-- Logs begin at Sat 2022-07-16 14:16:06 UTC, end at Sat 2022-07-16 15:58:02 UTC. -- Jul 16 14:47:56.192230 dv-cinder-836252-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 16 14:47:56.200857 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 16 14:47:56.203465 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Jul 16 14:47:56 2022] *** Jul 16 14:47:56.203606 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 16 14:47:56.203721 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 16 14:47:56.203833 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: nodename: dv-cinder-836252-5-iscsi Jul 16 14:47:56.203935 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: machine: x86_64 Jul 16 14:47:56.204038 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: clock source: unix Jul 16 14:47:56.204132 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: pcre jit disabled Jul 16 14:47:56.204231 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: detected number of CPU cores: 4 Jul 16 14:47:56.204331 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: current working directory: / Jul 16 14:47:56.204430 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: detected binary path: /usr/bin/uwsgi-core Jul 16 14:47:56.204542 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: your processes number limit is 47680 Jul 16 14:47:56.204647 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: your memory page size is 4096 bytes Jul 16 14:47:56.204743 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: detected max file descriptor number: 2048 Jul 16 14:47:56.204846 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: lock engine: pthread robust mutexes Jul 16 14:47:56.205105 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: thunder lock: enabled Jul 16 14:47:56.205399 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 16 14:47:56.205590 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 16 14:47:56.246624 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: Python main interpreter initialized at 0x5653c5e2a9c0 Jul 16 14:47:56.246624 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: python threads support enabled Jul 16 14:47:56.246624 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: your server socket listen backlog is limited to 100 connections Jul 16 14:47:56.246624 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: your mercy for graceful operations on workers is 90 seconds Jul 16 14:47:56.247014 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: mapped 403077 bytes (393 KB) for 2 cores Jul 16 14:47:56.247099 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: *** Operational MODE: preforking *** Jul 16 14:47:56.247344 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: *** uWSGI is running in multiple interpreter mode *** Jul 16 14:47:56.247344 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: spawned uWSGI master process (pid: 80247) Jul 16 14:47:56.247551 dv-cinder-836252-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 16 14:47:56.248539 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: spawned uWSGI worker 1 (pid: 80256, cores: 1) Jul 16 14:47:56.248539 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: spawned uWSGI worker 2 (pid: 80257, cores: 1) Jul 16 14:47:56.248689 dv-cinder-836252-5-iscsi devstack@keystone.service[80247]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 16 14:48:01.733984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 16 14:48:01.733984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: from cryptography.utils import int_from_bytes Jul 16 14:48:01.734786 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 16 14:48:01.734786 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: from cryptography.utils import int_from_bytes Jul 16 14:48:01.735237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 16 14:48:01.735237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: from cryptography.utils import int_from_bytes Jul 16 14:48:01.735348 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 16 14:48:01.735348 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: from cryptography.utils import int_from_bytes Jul 16 14:48:01.784354 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 16 14:48:01.784543 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 16 14:48:01.784636 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 16 14:48:01.784728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] command line args: [] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 16 14:48:01.784807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 16 14:48:01.784963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] command line args: [] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 16 14:48:01.785047 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 16 14:48:01.785131 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ================================================================================ {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 16 14:48:01.785237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 16 14:48:01.785367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] admin_token = **** {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.785452 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ================================================================================ {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 16 14:48:01.785606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] config_dir = [] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.785696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] admin_token = **** {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.785793 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.785877 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] config_dir = [] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.785955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] config_source = [] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.786037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.786115 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] debug = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.786222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] config_source = [] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.786308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.786570 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] debug = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.786570 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.786716 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.786868 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.787011 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.787116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.787225 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.787331 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.787460 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.787566 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.787677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] list_limit = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.787677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] log_config_append = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.787866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] list_limit = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.787968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.788082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] log_config_append = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.788190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] log_dir = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.788291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.788387 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] log_file = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.788489 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] log_dir = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.788489 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] log_file = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.788711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.788787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.788909 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.788990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.789062 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.789137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.789217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.789217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.789408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.789491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.789569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.789569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.789777 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.789903 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.789984 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.790057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.790128 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.790265 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.790378 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.790482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.790651 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.790738 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.790862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.791024 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.791100 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.791231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.791349 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.791970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.792060 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.792250 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.792551 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.792735 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] publish_errors = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.792817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.792965 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.793049 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] publish_errors = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.793198 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.793277 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.793356 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.793474 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.793588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.793663 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.793819 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.793911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.794050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] standard_threads = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.794133 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.794284 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] standard_threads = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.794392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.794514 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.794594 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.794754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.794830 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.794948 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] use_journal = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.795036 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.795186 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] use_json = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.795266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] use_journal = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.795363 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] use_stderr = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.795446 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] use_json = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.795446 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] use_stderr = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.795581 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] use_syslog = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.795710 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] use_syslog = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.795792 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.795870 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 14:48:01.795946 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.796065 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.796143 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.796306 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.796390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.796565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.796642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.796804 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.796882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.797049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.797126 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.797283 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.797361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.797515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.797590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.797753 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.797836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.797991 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.798067 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.798233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.798315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.798462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.798546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.798699 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.798775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.798949 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.799035 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.799193 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.799273 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.799436 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.799519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.799678 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.799760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.799911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.799989 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.800163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.800238 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.800403 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.800479 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.800624 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.800704 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.800866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.800948 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.801120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.801205 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.801364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.801448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.801609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.801686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.801835 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.801940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.802073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.802150 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.802301 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.802380 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.802530 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.802621 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.802764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.802842 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.803036 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.803117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.803282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.803396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.803512 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.803591 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.803741 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.803825 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.803986 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.804075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.804220 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.804336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.804435 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.804572 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.804680 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.804789 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.804887 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.804984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.805114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.805220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.805324 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.805411 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.805553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.805634 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.805781 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.805861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.806008 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.806100 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.806238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.806366 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.806473 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.806558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.806699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.806777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.807092 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.807179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.807326 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.807412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.807561 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.807672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.807780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.807896 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.808027 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.808112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.808254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.808332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.808490 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.808569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.808722 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.808800 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.808954 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.809037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.809185 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.809272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.809417 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.809494 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.809642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.809724 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.809880 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.809957 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.810109 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.810185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.810343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.810422 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.810570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.810655 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.810800 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.810946 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.811085 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.811173 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.811328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.811418 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.811558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.811643 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.811795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.811874 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.812020 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.812095 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.812275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.812362 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.812512 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.812589 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.812751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.812836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.812986 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.813063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.813220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.813296 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.813450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.813536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.813681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.813760 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.813913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.813990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.814146 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.814225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.814379 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.814457 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.814612 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.814689 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.814844 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.814939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.815097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.815176 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.815335 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.815432 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.815562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.815652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.815796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.815874 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.816043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.816121 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.816284 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.816369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.816517 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.816594 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.816749 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.816826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.816979 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.817064 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.817210 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.817288 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.817439 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.817515 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.817666 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.817751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.817891 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.817968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.818114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.818191 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.818339 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.818424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.818562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.818639 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.818789 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.818866 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.819041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.819128 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.819283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.819378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.819514 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.819592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.819747 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.819834 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.820001 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.820079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.820232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.820310 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.820459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] auth.external = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.820545 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.820684 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.820763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] auth.external = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.820910 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.820994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.821198 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.821286 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.821425 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] auth.password = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.821511 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.821649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] auth.token = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.821727 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] auth.password = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.821874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] auth.token = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.821957 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.822105 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.822191 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.822339 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.822416 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.822605 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.822687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.822836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.822945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.823116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.823202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.823374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.823458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.823607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.823694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] credential.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.823842 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] credential.caching = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.823921 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.824087 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.824166 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.824334 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.824420 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.824576 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.824653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.824807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.824890 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.825038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.825124 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.825270 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.825347 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.825532 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.825610 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.825765 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.825917 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.825998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.826077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.826249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.826328 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.826507 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.826597 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.826762 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.826852 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.827023 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.827102 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.827277 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.827367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.827509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] federation.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.827586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.827744 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.827829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] federation.caching = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.827987 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.828073 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.828228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.828307 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.828456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.828535 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.828688 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.828773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.828918 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.828996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.829157 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.829234 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.829387 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.829465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.829629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.829706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.829859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.829936 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.830102 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.830182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.830335 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.830413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.830567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] identity.caching = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.830651 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.830808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.830904 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.831063 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.831144 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.831294 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.831373 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.831527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.831656 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.831884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.832001 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.832116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.832195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.832357 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.832438 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.832604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.832683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.832838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.832917 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.833072 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.833150 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.833299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.833376 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.833528 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.833605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.833771 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.833852 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.834013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.834091 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.834257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.834335 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.834498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.834577 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.834746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.834823 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.835022 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.835111 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.835258 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.835338 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.835489 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.835567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.835720 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.835842 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.835933 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.836013 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.836168 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.836248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.836403 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.836481 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.836638 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.836718 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.836884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.836964 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.837112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.837189 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.837349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.837676 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.837842 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.837920 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.838080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.838663 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.838872 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.839011 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.839138 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.839218 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.839383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.839479 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.839611 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.839792 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.839871 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.840024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.840110 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.840263 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.840343 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.840499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.840578 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.840737 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.840818 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.840972 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.841053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.841206 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.841285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.841453 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.841532 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.841696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.841776 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.841924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.842002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.842149 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.842226 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.842393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.842473 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.842638 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.842720 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.842871 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.842970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.843130 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.843211 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.843365 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.843444 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.843591 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.843669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.user = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.843828 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.843915 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.844083 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.844163 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.844311 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.844390 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.844554 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.844633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.844794 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.844871 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.845037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.845116 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.845267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.845347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.845492 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.845570 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.845723 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.845801 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.845953 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.846032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.846184 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.846262 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.846409 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.846486 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.846650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.846729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.846936 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.847018 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.847149 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.847228 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.847391 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.847472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.847630 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.847708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.847856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.847933 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.848096 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.848175 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.848328 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.848407 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.848559 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.848638 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.848790 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.848870 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.849020 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.849098 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.849256 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.849334 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.849482 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.849557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.849720 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.849798 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.849969 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.850048 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.850193 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.850272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.850433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.850511 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.850664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.850821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.850944 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.851021 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.851137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.851217 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.851376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.851455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.851612 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.851693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.851838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.851916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.852070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.852186 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.852307 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] resource.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.852433 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] resource.caching = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.852557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.852683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.852799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.852930 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.853038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.853166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.853282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.853416 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.853525 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.853664 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.854390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.854530 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.854657 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.854792 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.854959 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.855073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.855166 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.855242 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.855314 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] role.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.855386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] role.caching = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.855552 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] role.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.855641 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] role.driver = sql {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.855780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.855860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.856024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.856110 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.856266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.856355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.856508 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.856589 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.856750 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.856830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.856993 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.857081 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.857235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.857314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.857475 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.857555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.857719 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.857806 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.857945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.858023 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.858186 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.858264 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.858426 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.858512 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.858668 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.858748 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.858935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.859028 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.859333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.859421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.859568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.859646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.859812 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.859891 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.860057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.860143 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.860300 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.860379 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.860545 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.860623 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.860773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.860853 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.861037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.861159 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.861276 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.861351 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.861496 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.861577 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.861723 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.861802 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.861950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.862029 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.862178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.862258 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.862421 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.862498 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.862670 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.862749 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.862939 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.863052 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.863175 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.863263 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.863408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.863487 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.863633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.863712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.863863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] token.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.863943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] token.caching = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.864097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.864176 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.864340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.864421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.864573 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.864651 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.864820 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.864898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.865061 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.865142 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.865295 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.865374 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.865531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.865609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.865770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.865851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.866013 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.866092 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.866243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.866321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.866477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.866558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.866708 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.866787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.866974 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.867061 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.867219 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.867299 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.867444 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.867523 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.867682 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.867759 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.867965 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.868044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.868221 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] database.connection = **** {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.868308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.connection = **** {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.868461 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.868553 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.868703 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.868783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.868936 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.869088 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.869166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.869238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.869433 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.869520 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.869670 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.869759 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.869906 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.869986 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.870141 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.870330 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.870409 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.870593 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.870674 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.870832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.870926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.871103 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.871182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.871341 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.871422 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.871573 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.871651 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.871817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.871896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.872046 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.872134 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.872285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.872371 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.872519 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.872600 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.872681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 16 14:48:01.872756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 14:48:01.872916 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 16 14:48:01.904391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:01.904532 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:01.904622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:01.904734 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:01.904821 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:01.904960 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:01.913254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:01.913406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:01.942261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:01.942512 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:01.942602 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:01.942678 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.086977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.087783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.136857 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.137083 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.138597 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.138833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.202206 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.202427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.202547 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.202639 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.202781 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.202869 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.203034 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.203119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.203255 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.203339 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.203471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.203685 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.239401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.239607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.239726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.239870 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.239981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.240060 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.240133 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.240242 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.240337 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.240456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.240539 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.240670 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.240752 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.240887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.252266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.252573 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.266029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.266290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.266468 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.266684 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.266833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.266939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.267095 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.267183 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.267319 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80256) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.267415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.267536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.267751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.267968 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.268177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80257) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 16 14:48:02.285201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.285456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 16 14:48:02.285741 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.285826 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.285937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 16 14:48:02.286042 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 16 14:48:02.286200 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.286284 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.286450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 16 14:48:02.286533 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.286699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.286782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.287022 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.287113 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.287274 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.287532 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.298083 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.298395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.298610 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.298879 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.299165 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.299445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.299705 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.299981 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.307642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.308869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80257) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.311158 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.311435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.311712 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.312107 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.312190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80257) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.312387 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.312484 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.312644 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.312728 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.312909 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.313096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.313189 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.313382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.313487 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.313639 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.313726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.313904 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.313988 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.314174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.314441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.314710 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.314997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.341620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.342771 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.343190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.343463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.344438 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80257) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.344715 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.348692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.348964 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.349235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.349988 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80257) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.350263 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.350537 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.356631 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.356907 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.358292 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80257) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.358577 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.361526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.361806 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.362082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.362358 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.362628 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.362923 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.363182 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.363265 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80257) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.363446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.363534 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.363790 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.364066 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.364334 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.364601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.364866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.365130 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.382650 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.382953 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.384681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.384976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.388113 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.390224 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80257) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.391909 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.392195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.393901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.394191 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.397174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.399239 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80257) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.400847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.402910 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80257) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.404581 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.404869 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.405153 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.405437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.405714 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.405993 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.406699 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.407013 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.407289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.407573 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.407843 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.408126 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.424880 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.425247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.425510 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.425774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.427229 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80257) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.427600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.427863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.428131 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.434283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.436015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.436280 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.436760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.438461 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.438730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.439303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.439597 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.439870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.440151 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.440428 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.441810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80257) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.442086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.442362 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.442646 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.442940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.454702 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.454999 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.455272 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.455545 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.455813 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.456088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.456350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.456884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80257) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.457159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.457434 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.457709 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.458090 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.458369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.458639 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.476157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.478567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80257) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.479462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.481896 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80257) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.482753 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.484356 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.485238 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.485977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.486251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.486525 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.486836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.488498 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80257) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.488769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.489047 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.493908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.496319 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.497058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.498653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80257) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.499612 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.499895 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.500170 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.500445 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.501923 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.502209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.502486 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.502755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.509657 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.509944 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.511010 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.512590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80257) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.512875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.513149 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.518873 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80256) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.519164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.519441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.519707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.519981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.520261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.520529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.520797 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.521025 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80257) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 16 14:48:02.521125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.521292 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.521376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.521568 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.521652 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.521831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.521913 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.522093 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.522175 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.522351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80256) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.522435 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.522627 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.522920 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.523200 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.523467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.523731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.523993 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.524250 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.524502 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80257) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 16 14:48:02.560956 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5653c5e2a9c0 pid: 80256 (default app) Jul 16 14:48:02.563117 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5653c5e2a9c0 pid: 80257 (default app) Jul 16 14:48:02.564418 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfb8d0-39b1-474d-846d-2e9bdc867fdf None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:02.564707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfb8d0-39b1-474d-846d-2e9bdc867fdf None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:02.564987 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfb8d0-39b1-474d-846d-2e9bdc867fdf None None] PATH_INFO: `/v3` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:02.569183 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 791 bytes} [Sat Jul 16 14:48:02 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:11.274273 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8f3a5-7042-423d-94e2-3ccd37b36b96 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:11.274747 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8f3a5-7042-423d-94e2-3ccd37b36b96 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:11.274747 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8f3a5-7042-423d-94e2-3ccd37b36b96 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:11.280251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1/2] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:48:11.285918 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e841f1-ddc7-48b5-a978-8a259c41b9c9 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:11.286347 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e841f1-ddc7-48b5-a978-8a259c41b9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:11.286448 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e841f1-ddc7-48b5-a978-8a259c41b9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:11.353951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: INFO dbcounter [None req-e6e841f1-ddc7-48b5-a978-8a259c41b9c9 None None] Registered counter for database keystone Jul 16 14:48:11.375543 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG oslo_db.sqlalchemy.engines [None req-e6e841f1-ddc7-48b5-a978-8a259c41b9c9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80256) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 16 14:48:11.381721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writer thread running {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 16 14:48:11.558503 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG passlib.handlers.bcrypt [None req-e6e841f1-ddc7-48b5-a978-8a259c41b9c9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80256) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 16 14:48:11.558834 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG passlib.handlers.bcrypt [None req-e6e841f1-ddc7-48b5-a978-8a259c41b9c9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80256) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 16 14:48:11.613951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-e6e841f1-ddc7-48b5-a978-8a259c41b9c9 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:11.615592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e6e841f1-ddc7-48b5-a978-8a259c41b9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:11.737971 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 2/3] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 16 14:48:11.902434 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8f3a5-7042-423d-94e2-3ccd37b36b96 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:11.973640 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: INFO dbcounter [None req-a3f8f3a5-7042-423d-94e2-3ccd37b36b96 None None] Registered counter for database keystone Jul 16 14:48:11.993538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG oslo_db.sqlalchemy.engines [None req-a3f8f3a5-7042-423d-94e2-3ccd37b36b96 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80257) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 16 14:48:12.000114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writer thread running {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 16 14:48:12.127085 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8f3a5-7042-423d-94e2-3ccd37b36b96 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:12.131568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2001c87-6063-4894-bd23-433731b95ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:12.132592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2001c87-6063-4894-bd23-433731b95ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:12.132877 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2001c87-6063-4894-bd23-433731b95ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:12.133156 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2001c87-6063-4894-bd23-433731b95ff7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:12.142028 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2001c87-6063-4894-bd23-433731b95ff7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:12.196281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d2001c87-6063-4894-bd23-433731b95ff7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 16 14:48:12.196552 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2001c87-6063-4894-bd23-433731b95ff7 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:12.207516 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 2/4] 10.222.0.27 () {58 vars in 1324 bytes} [Sat Jul 16 14:48:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 16 14:48:12.214674 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e841f1-ddc7-48b5-a978-8a259c41b9c9 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:12.230192 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e841f1-ddc7-48b5-a978-8a259c41b9c9 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:12.234718 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a1155-f139-402e-a95d-f50b31f61d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:12.235877 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a1155-f139-402e-a95d-f50b31f61d7d admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:12.236167 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a1155-f139-402e-a95d-f50b31f61d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:12.236446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a1155-f139-402e-a95d-f50b31f61d7d admin admin] PATH_INFO: `/v3/projects/094ab8f948f540f7a205e832cbae4778` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:12.240693 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a1155-f139-402e-a95d-f50b31f61d7d admin admin] RBAC: Authorizing `identity:get_project(project_id=094ab8f948f540f7a205e832cbae4778)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:12.295319 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8e7a1155-f139-402e-a95d-f50b31f61d7d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 16 14:48:12.295944 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a1155-f139-402e-a95d-f50b31f61d7d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:12.300454 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 3/5] 10.222.0.27 () {56 vars in 1207 bytes} [Sat Jul 16 14:48:12 2022] GET /identity/v3/projects/094ab8f948f540f7a205e832cbae4778 => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:48:14.299746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6fd03-249b-4159-97c9-b9fd723232a5 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:14.300243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6fd03-249b-4159-97c9-b9fd723232a5 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:14.300243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6fd03-249b-4159-97c9-b9fd723232a5 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:14.304325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 3/6] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:14.310218 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff84d1-9f57-41fb-8bb7-327f932e1222 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:14.310641 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff84d1-9f57-41fb-8bb7-327f932e1222 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:14.310778 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff84d1-9f57-41fb-8bb7-327f932e1222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:14.329970 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-31ff84d1-9f57-41fb-8bb7-327f932e1222 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:14.331543 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-31ff84d1-9f57-41fb-8bb7-327f932e1222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:14.340647 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 4/7] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 16 14:48:14.363247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6fd03-249b-4159-97c9-b9fd723232a5 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:14.375286 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6fd03-249b-4159-97c9-b9fd723232a5 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:14.379453 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693c484-4dcd-455f-9bed-1fe3140fa3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:14.380072 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693c484-4dcd-455f-9bed-1fe3140fa3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:14.380355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693c484-4dcd-455f-9bed-1fe3140fa3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:14.380631 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693c484-4dcd-455f-9bed-1fe3140fa3f2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:14.388462 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693c484-4dcd-455f-9bed-1fe3140fa3f2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:14.390234 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693c484-4dcd-455f-9bed-1fe3140fa3f2 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:14.399224 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 4/8] 10.222.0.27 () {58 vars in 1324 bytes} [Sat Jul 16 14:48:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 14:48:14.405743 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff84d1-9f57-41fb-8bb7-327f932e1222 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:14.414916 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff84d1-9f57-41fb-8bb7-327f932e1222 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:14.419891 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91353f4-d880-49d2-b7c6-325f78a4be5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:14.420767 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91353f4-d880-49d2-b7c6-325f78a4be5d admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:14.421056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91353f4-d880-49d2-b7c6-325f78a4be5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:14.421333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91353f4-d880-49d2-b7c6-325f78a4be5d admin admin] PATH_INFO: `/v3/users/2e3ec4696fd447c89f8d404bb4540c6c` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:14.422735 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91353f4-d880-49d2-b7c6-325f78a4be5d admin admin] RBAC: Authorizing `identity:get_user(user_id=2e3ec4696fd447c89f8d404bb4540c6c)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:14.424608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91353f4-d880-49d2-b7c6-325f78a4be5d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:14.426211 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 5/9] 10.222.0.27 () {56 vars in 1198 bytes} [Sat Jul 16 14:48:14 2022] GET /identity/v3/users/2e3ec4696fd447c89f8d404bb4540c6c => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:16.534389 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f883212-ab67-4357-86a7-3d13a0383c5b None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:16.762716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f883212-ab67-4357-86a7-3d13a0383c5b None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:16.762716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f883212-ab67-4357-86a7-3d13a0383c5b None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:16.762716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 5/10] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:16.763025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62262a-750d-440c-a03c-0c861c2c889e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:16.763025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62262a-750d-440c-a03c-0c861c2c889e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:16.763025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62262a-750d-440c-a03c-0c861c2c889e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:16.763025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-1f62262a-750d-440c-a03c-0c861c2c889e None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:16.763025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1f62262a-750d-440c-a03c-0c861c2c889e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:16.763713 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db7432-fb00-4be3-896e-63c6b8ba0f81 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:16.763713 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db7432-fb00-4be3-896e-63c6b8ba0f81 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:16.763713 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db7432-fb00-4be3-896e-63c6b8ba0f81 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:16.763713 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 6/11] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:48:16.763713 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ff688-99a7-47a0-b068-f560e96d387b None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:16.763713 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ff688-99a7-47a0-b068-f560e96d387b None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:16.764146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ff688-99a7-47a0-b068-f560e96d387b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:16.764146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG passlib.handlers.bcrypt [None req-ba6ff688-99a7-47a0-b068-f560e96d387b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80257) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 16 14:48:16.764146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG passlib.handlers.bcrypt [None req-ba6ff688-99a7-47a0-b068-f560e96d387b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80257) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 16 14:48:16.764328 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 6/12] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:16.764328 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62262a-750d-440c-a03c-0c861c2c889e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:16.764465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-ba6ff688-99a7-47a0-b068-f560e96d387b None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:16.764465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ba6ff688-99a7-47a0-b068-f560e96d387b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:16.764591 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:16.786881 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62262a-750d-440c-a03c-0c861c2c889e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:16.793586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 7/13] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:16.816316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ff688-99a7-47a0-b068-f560e96d387b None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:16.824047 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8381326-a86b-47c6-a560-c6e36a5b649d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:16.824858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8381326-a86b-47c6-a560-c6e36a5b649d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:16.825149 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8381326-a86b-47c6-a560-c6e36a5b649d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:16.825428 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8381326-a86b-47c6-a560-c6e36a5b649d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:16.837133 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8381326-a86b-47c6-a560-c6e36a5b649d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:16.849040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-a8381326-a86b-47c6-a560-c6e36a5b649d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:48:16.849040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:16.849040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:16.849040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:16.849040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:16.849040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:16.849040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:16.849040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:16.849040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:16.849040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:16.849040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:16.849040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:16.849040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:16.849040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:16.849040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:16.849040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:16.850038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:16.850038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:16.850038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:16.850038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:16.850038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:16.850038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:16.850038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:16.850038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:16.850038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:16.850038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:16.850038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:16.850038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:16.850038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:16.850038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:16.850038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:16.850038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:16.850994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:16.850994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:16.850994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:16.850994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:16.850994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:16.850994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:16.850994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:16.850994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:16.850994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:16.850994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:16.850994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:48:16.850994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:48:16.852083 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 7/14] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:48:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:48:16.856217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8381326-a86b-47c6-a560-c6e36a5b649d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:16.923619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ff688-99a7-47a0-b068-f560e96d387b None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:16.926751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:16.942786 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:16.960638 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8381326-a86b-47c6-a560-c6e36a5b649d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:16.964592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47919466-d7a6-452f-b7f5-b764d51d0b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:16.965594 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47919466-d7a6-452f-b7f5-b764d51d0b77 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:16.965882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47919466-d7a6-452f-b7f5-b764d51d0b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:16.966157 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47919466-d7a6-452f-b7f5-b764d51d0b77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:16.996872 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ba741-504a-4e67-9a69-e9f73008e065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:16.997554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ba741-504a-4e67-9a69-e9f73008e065 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:16.997847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ba741-504a-4e67-9a69-e9f73008e065 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:16.998454 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ba741-504a-4e67-9a69-e9f73008e065 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:16.999890 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ba741-504a-4e67-9a69-e9f73008e065 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:17.000479 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ba741-504a-4e67-9a69-e9f73008e065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:17.002406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ba741-504a-4e67-9a69-e9f73008e065 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:17.014657 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 8/15] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:48:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:17.020436 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ba741-504a-4e67-9a69-e9f73008e065 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:17.037005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47919466-d7a6-452f-b7f5-b764d51d0b77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:17.038440 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47919466-d7a6-452f-b7f5-b764d51d0b77 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:17.130645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ba741-504a-4e67-9a69-e9f73008e065 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:17.144823 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 8/16] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:48:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 16 14:48:17.150502 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47919466-d7a6-452f-b7f5-b764d51d0b77 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:17.153548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:17.168886 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac535c-bc34-4c39-83ca-a63b97799b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:17.169776 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac535c-bc34-4c39-83ca-a63b97799b4d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:17.170059 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac535c-bc34-4c39-83ca-a63b97799b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:17.170334 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac535c-bc34-4c39-83ca-a63b97799b4d None admin] PATH_INFO: `/v3/users/2e3ec4696fd447c89f8d404bb4540c6c` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:17.171621 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac535c-bc34-4c39-83ca-a63b97799b4d None admin] RBAC: Authorizing `identity:get_user(user_id=2e3ec4696fd447c89f8d404bb4540c6c)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:17.173285 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac535c-bc34-4c39-83ca-a63b97799b4d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:17.174760 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 9/17] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:17 2022] GET /identity/v3/users/2e3ec4696fd447c89f8d404bb4540c6c => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:48:17.180260 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac535c-bc34-4c39-83ca-a63b97799b4d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:17.197001 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:17.262796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47919466-d7a6-452f-b7f5-b764d51d0b77 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:17.297224 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac535c-bc34-4c39-83ca-a63b97799b4d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:17.302605 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d9e1e-cb31-45d2-a962-8e8bac72e3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:17.303442 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d9e1e-cb31-45d2-a962-8e8bac72e3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:17.303717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d9e1e-cb31-45d2-a962-8e8bac72e3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:17.303986 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d9e1e-cb31-45d2-a962-8e8bac72e3f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:17.327616 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d9e1e-cb31-45d2-a962-8e8bac72e3f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:17.329109 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d9e1e-cb31-45d2-a962-8e8bac72e3f5 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:17.335472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3a845-d5e1-469b-ac48-0d483355c215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:17.336559 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3a845-d5e1-469b-ac48-0d483355c215 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:17.336845 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3a845-d5e1-469b-ac48-0d483355c215 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:17.337118 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3a845-d5e1-469b-ac48-0d483355c215 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:17.339555 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b3a845-d5e1-469b-ac48-0d483355c215 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:17.341116 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b3a845-d5e1-469b-ac48-0d483355c215 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:17.342801 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 10/18] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:48:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:48:17.348545 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3a845-d5e1-469b-ac48-0d483355c215 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:17.351406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-e96d9e1e-cb31-45d2-a962-8e8bac72e3f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:48:17.351406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:17.351406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:17.351406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:17.351406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:17.351406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:17.351406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:17.351406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:17.351406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:17.351406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:17.351406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:17.351406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:17.351406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:48:17.351406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:48:17.351406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:48:17.352693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:48:17.352693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:17.352693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:17.352693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:17.352693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:17.352693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:17.352693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:17.352693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:17.352693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:17.352693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:17.352693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:17.352693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:17.352693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:17.352693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:17.352693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:17.352693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:17.352693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:17.353787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:48:17.353787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:48:17.353787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:48:17.353787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:48:17.353787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 9/19] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:48:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:48:17.360409 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d9e1e-cb31-45d2-a962-8e8bac72e3f5 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:17.428355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:17.442294 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:17.463417 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3a845-d5e1-469b-ac48-0d483355c215 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:17.479239 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d9e1e-cb31-45d2-a962-8e8bac72e3f5 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:17.512975 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e8ce9-d4d2-495a-ba80-4a0b529f8c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:17.513646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e8ce9-d4d2-495a-ba80-4a0b529f8c72 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:17.513956 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e8ce9-d4d2-495a-ba80-4a0b529f8c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:17.514270 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e8ce9-d4d2-495a-ba80-4a0b529f8c72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:17.515301 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab3798-ba86-47d6-b888-c014dfc36b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:17.515449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e8ce9-d4d2-495a-ba80-4a0b529f8c72 None admin] RBAC: Adding query filter params (role.id=0efeff46573247f3b9f1e2dec2968672, scope.domain.id=default, user.id=2e3ec4696fd447c89f8d404bb4540c6c) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:17.515882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e8ce9-d4d2-495a-ba80-4a0b529f8c72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:17.515982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab3798-ba86-47d6-b888-c014dfc36b6f None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:17.516225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab3798-ba86-47d6-b888-c014dfc36b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:17.516495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab3798-ba86-47d6-b888-c014dfc36b6f None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:17.517222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab3798-ba86-47d6-b888-c014dfc36b6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:17.517530 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab3798-ba86-47d6-b888-c014dfc36b6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:17.517844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e8ce9-d4d2-495a-ba80-4a0b529f8c72 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:17.518918 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab3798-ba86-47d6-b888-c014dfc36b6f None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:17.526883 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 11/20] 10.222.0.27 () {56 vars in 1322 bytes} [Sat Jul 16 14:48:17 2022] GET /identity/v3/role_assignments?user.id=2e3ec4696fd447c89f8d404bb4540c6c&scope.domain.id=default&role.id=0efeff46573247f3b9f1e2dec2968672 => generated 233 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:48:17.538963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 10/21] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:48:20.203776 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae7db2-5bd8-4ce9-b17c-f59eb681f664 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:20.204484 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae7db2-5bd8-4ce9-b17c-f59eb681f664 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:20.204484 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae7db2-5bd8-4ce9-b17c-f59eb681f664 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:20.205832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 12/22] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:48:20.212204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf584abd-5de3-4766-a2b8-279e7b9ad280 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:20.214074 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf584abd-5de3-4766-a2b8-279e7b9ad280 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:20.214837 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf584abd-5de3-4766-a2b8-279e7b9ad280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:20.257615 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-cf584abd-5de3-4766-a2b8-279e7b9ad280 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:20.260070 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-cf584abd-5de3-4766-a2b8-279e7b9ad280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:20.372016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 11/23] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 14:48:20.407282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae7db2-5bd8-4ce9-b17c-f59eb681f664 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:20.575545 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae7db2-5bd8-4ce9-b17c-f59eb681f664 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:20.641809 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d555e-7dff-44d3-b157-4319616e0ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:20.642617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d555e-7dff-44d3-b157-4319616e0ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:20.642923 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d555e-7dff-44d3-b157-4319616e0ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:20.643207 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d555e-7dff-44d3-b157-4319616e0ff0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:20.663223 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d555e-7dff-44d3-b157-4319616e0ff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:20.664729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d555e-7dff-44d3-b157-4319616e0ff0 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:20.679267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:20.695664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-3d5d555e-7dff-44d3-b157-4319616e0ff0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:48:20.695664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:20.695664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:20.695664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:20.695664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:20.695664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:20.695664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:20.695664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:20.695664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:20.695664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:20.695664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:20.695664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:20.695664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:48:20.695664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:48:20.695664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:48:20.703529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:48:20.703529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:20.703529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:20.703529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:20.703529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:20.703529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:20.703529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:20.703529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:20.703529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:20.703529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:20.703529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:20.703529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:20.703529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:20.703529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:20.703529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:20.703529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:20.703529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:20.705071 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:48:20.705071 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:48:20.705071 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:48:20.705071 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:48:20.705071 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 13/24] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:48:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:20.708071 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf584abd-5de3-4766-a2b8-279e7b9ad280 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:20.826376 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:20.850294 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571e591-a994-4d63-82d6-fc6a2b2906b0 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:20.856699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571e591-a994-4d63-82d6-fc6a2b2906b0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:20.856699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571e591-a994-4d63-82d6-fc6a2b2906b0 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:20.856699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 14/25] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:48:20.868504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457c00a-83b3-4c4b-92d2-313f0f82ad4f None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:20.868738 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457c00a-83b3-4c4b-92d2-313f0f82ad4f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:20.869769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457c00a-83b3-4c4b-92d2-313f0f82ad4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:20.918989 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-6457c00a-83b3-4c4b-92d2-313f0f82ad4f None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:20.920261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6457c00a-83b3-4c4b-92d2-313f0f82ad4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:20.966684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf584abd-5de3-4766-a2b8-279e7b9ad280 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:21.037183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab85e2a-d361-41c9-a997-8c131e036403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:21.038858 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab85e2a-d361-41c9-a997-8c131e036403 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:21.039780 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab85e2a-d361-41c9-a997-8c131e036403 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:21.040538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab85e2a-d361-41c9-a997-8c131e036403 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:21.042389 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab85e2a-d361-41c9-a997-8c131e036403 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:21.043257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab85e2a-d361-41c9-a997-8c131e036403 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:21.046179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab85e2a-d361-41c9-a997-8c131e036403 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:21.069673 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 15/26] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:21.070733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70242b73-3805-470e-9425-cf9fe749d0c9 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:21.071058 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70242b73-3805-470e-9425-cf9fe749d0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:21.071328 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70242b73-3805-470e-9425-cf9fe749d0c9 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:21.073395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 12/28] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:21.073641 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 16/28] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:21.079504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0e28b-5a21-45a7-af00-a4b7229b3b97 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:21.079703 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0e28b-5a21-45a7-af00-a4b7229b3b97 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:21.079979 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0e28b-5a21-45a7-af00-a4b7229b3b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:21.093137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab85e2a-d361-41c9-a997-8c131e036403 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:21.116055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-69a0e28b-5a21-45a7-af00-a4b7229b3b97 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:21.117270 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-69a0e28b-5a21-45a7-af00-a4b7229b3b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:21.219138 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 17/29] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:21.220073 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0e28b-5a21-45a7-af00-a4b7229b3b97 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:21.286920 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab85e2a-d361-41c9-a997-8c131e036403 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:21.287659 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:21.297394 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:21.326975 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8d2e0-e006-414a-8533-7bb94c7b730d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:21.327697 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8d2e0-e006-414a-8533-7bb94c7b730d None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:21.328021 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8d2e0-e006-414a-8533-7bb94c7b730d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:21.328302 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8d2e0-e006-414a-8533-7bb94c7b730d None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:21.329318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8d2e0-e006-414a-8533-7bb94c7b730d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:21.330940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8d2e0-e006-414a-8533-7bb94c7b730d None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:21.343850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0e28b-5a21-45a7-af00-a4b7229b3b97 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:21.383499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d9909-960f-4e22-bf3d-471c83455f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:21.384283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d9909-960f-4e22-bf3d-471c83455f92 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:21.384645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d9909-960f-4e22-bf3d-471c83455f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:21.384902 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d9909-960f-4e22-bf3d-471c83455f92 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:21.386138 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d9909-960f-4e22-bf3d-471c83455f92 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:21.387610 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d9909-960f-4e22-bf3d-471c83455f92 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:21.555056 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 13/30] 10.222.0.27 () {60 vars in 1141 bytes} [Sat Jul 16 14:48:21 2022] POST /identity/v3/projects => generated 301 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 14:48:21.557603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355df9e0-e95f-4f62-be00-aad9fed9dbab None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:21.558179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355df9e0-e95f-4f62-be00-aad9fed9dbab None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:21.558567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355df9e0-e95f-4f62-be00-aad9fed9dbab None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:21.564464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 14/31] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:21.565580 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355df9e0-e95f-4f62-be00-aad9fed9dbab None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:21.589511 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 18/32] 10.222.0.27 () {60 vars in 1131 bytes} [Sat Jul 16 14:48:21 2022] POST /identity/v3/roles => generated 219 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:48:21.591254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb121265-de36-41fb-9204-b7c60aeedb3b None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:21.591600 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb121265-de36-41fb-9204-b7c60aeedb3b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:21.591882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb121265-de36-41fb-9204-b7c60aeedb3b None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:21.593697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 19/33] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:21.595056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9cba6-1ee0-47fd-a9d6-c1ae81958627 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:21.595351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9cba6-1ee0-47fd-a9d6-c1ae81958627 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:21.595606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9cba6-1ee0-47fd-a9d6-c1ae81958627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:21.626741 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-88f9cba6-1ee0-47fd-a9d6-c1ae81958627 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:21.629249 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-88f9cba6-1ee0-47fd-a9d6-c1ae81958627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:21.769678 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355df9e0-e95f-4f62-be00-aad9fed9dbab None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:21.806174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 20/34] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:21.808020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e54de7-deb4-45b0-9608-74677c87b110 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:21.808403 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e54de7-deb4-45b0-9608-74677c87b110 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:21.808628 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e54de7-deb4-45b0-9608-74677c87b110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:21.849123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a7e54de7-deb4-45b0-9608-74677c87b110 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:21.850438 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a7e54de7-deb4-45b0-9608-74677c87b110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:21.870448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a6916-a477-42df-8c54-6b2180f52dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:21.871419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a6916-a477-42df-8c54-6b2180f52dfa None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:21.871708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a6916-a477-42df-8c54-6b2180f52dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:21.871987 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a6916-a477-42df-8c54-6b2180f52dfa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:21.895577 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a6916-a477-42df-8c54-6b2180f52dfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:21.914157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:21.914383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-274a6916-a477-42df-8c54-6b2180f52dfa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:48:21.914383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:21.914383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:21.914383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:21.914383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:21.914383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:21.914383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:21.914383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:21.914383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:21.914383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:21.914383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:21.914383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:21.914383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:21.914383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:21.914383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:21.914383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:21.916567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:21.916567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:21.916567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:21.916567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:21.916567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:21.916567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:21.916567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:21.916567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:21.916567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:21.916567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:21.916567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:21.916567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:21.916567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:21.916567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:21.916567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:21.916567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:21.917996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:21.917996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:21.917996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:21.917996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:21.917996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:21.917996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:21.917996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:21.917996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:21.917996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:21.917996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:21.917996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:48:21.917996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:48:21.917996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 15/35] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:48:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:21.919187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a6916-a477-42df-8c54-6b2180f52dfa None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:21.957337 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:21.984795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 21/36] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:21.987106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e54de7-deb4-45b0-9608-74677c87b110 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:22.059504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a6916-a477-42df-8c54-6b2180f52dfa None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:22.121237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cddb76-9654-4978-9bf8-fdc8b2a0806b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:22.122303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cddb76-9654-4978-9bf8-fdc8b2a0806b None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:22.122730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cddb76-9654-4978-9bf8-fdc8b2a0806b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:22.123225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cddb76-9654-4978-9bf8-fdc8b2a0806b None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:22.124314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cddb76-9654-4978-9bf8-fdc8b2a0806b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:22.126948 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cddb76-9654-4978-9bf8-fdc8b2a0806b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:22.137556 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e54de7-deb4-45b0-9608-74677c87b110 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:22.175201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec85eb-493f-48ab-a317-5979a0ba36b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:22.175887 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec85eb-493f-48ab-a317-5979a0ba36b2 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:22.176174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec85eb-493f-48ab-a317-5979a0ba36b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:22.176453 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec85eb-493f-48ab-a317-5979a0ba36b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:22.177229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec85eb-493f-48ab-a317-5979a0ba36b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:22.177539 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec85eb-493f-48ab-a317-5979a0ba36b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:22.179110 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec85eb-493f-48ab-a317-5979a0ba36b2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:22.189142 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 22/37] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:48:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:22.190029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec85eb-493f-48ab-a317-5979a0ba36b2 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:22.235766 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:22.243093 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 16/38] 10.222.0.27 () {60 vars in 1131 bytes} [Sat Jul 16 14:48:21 2022] POST /identity/v3/roles => generated 225 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:48:22.243666 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cddb76-9654-4978-9bf8-fdc8b2a0806b None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:22.343230 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec85eb-493f-48ab-a317-5979a0ba36b2 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:22.382853 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:22.403678 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3737a-7447-4f81-91ce-1f4f6dd8c870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:22.404761 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3737a-7447-4f81-91ce-1f4f6dd8c870 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:22.405208 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3737a-7447-4f81-91ce-1f4f6dd8c870 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:22.405667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3737a-7447-4f81-91ce-1f4f6dd8c870 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:22.406683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3737a-7447-4f81-91ce-1f4f6dd8c870 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:22.409060 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3737a-7447-4f81-91ce-1f4f6dd8c870 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:22.424749 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cddb76-9654-4978-9bf8-fdc8b2a0806b None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:22.469605 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000a511-e542-481a-afae-6038d9fb8dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:22.471050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000a511-e542-481a-afae-6038d9fb8dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:22.471512 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000a511-e542-481a-afae-6038d9fb8dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:22.471950 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000a511-e542-481a-afae-6038d9fb8dc9 None admin] PATH_INFO: `/v3/users/2e3ec4696fd447c89f8d404bb4540c6c` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:22.474556 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000a511-e542-481a-afae-6038d9fb8dc9 None admin] RBAC: Authorizing `identity:get_user(user_id=2e3ec4696fd447c89f8d404bb4540c6c)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:22.476429 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000a511-e542-481a-afae-6038d9fb8dc9 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:22.480719 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 17/39] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:22 2022] GET /identity/v3/users/2e3ec4696fd447c89f8d404bb4540c6c => generated 247 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:22.480719 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfd081-b521-4e39-97dc-b2c607075231 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:22.480719 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfd081-b521-4e39-97dc-b2c607075231 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:22.481086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfd081-b521-4e39-97dc-b2c607075231 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:22.482690 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 18/40] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:22.487347 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 23/41] 10.222.0.27 () {60 vars in 1131 bytes} [Sat Jul 16 14:48:22 2022] POST /identity/v3/roles => generated 223 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:48:22.491760 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3737a-7447-4f81-91ce-1f4f6dd8c870 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:22.491990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2150d-4ea9-4915-9bab-a8ba43f25b68 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:22.492318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2150d-4ea9-4915-9bab-a8ba43f25b68 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:22.492469 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2150d-4ea9-4915-9bab-a8ba43f25b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:22.516074 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-c3a2150d-4ea9-4915-9bab-a8ba43f25b68 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:22.518021 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c3a2150d-4ea9-4915-9bab-a8ba43f25b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:22.613000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 19/42] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:22.637160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2150d-4ea9-4915-9bab-a8ba43f25b68 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:22.763932 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:22.769139 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3737a-7447-4f81-91ce-1f4f6dd8c870 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:22.781403 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:22.808643 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10242dbf-c8a3-4e1d-bcfb-e3f29bb5e7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:22.808860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2150d-4ea9-4915-9bab-a8ba43f25b68 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:22.809600 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10242dbf-c8a3-4e1d-bcfb-e3f29bb5e7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:22.809954 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10242dbf-c8a3-4e1d-bcfb-e3f29bb5e7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:22.810338 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10242dbf-c8a3-4e1d-bcfb-e3f29bb5e7a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:22.811843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10242dbf-c8a3-4e1d-bcfb-e3f29bb5e7a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:22.813445 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10242dbf-c8a3-4e1d-bcfb-e3f29bb5e7a3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:22.815180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 24/43] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:48:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:22.821026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10242dbf-c8a3-4e1d-bcfb-e3f29bb5e7a3 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:22.849499 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2faf0a-f890-4965-9ed5-f8ee079d3dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:22.850351 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2faf0a-f890-4965-9ed5-f8ee079d3dff None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:22.850630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2faf0a-f890-4965-9ed5-f8ee079d3dff None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:22.850927 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2faf0a-f890-4965-9ed5-f8ee079d3dff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:22.852147 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2faf0a-f890-4965-9ed5-f8ee079d3dff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:22.854725 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2faf0a-f890-4965-9ed5-f8ee079d3dff None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:22.856594 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 20/44] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:48:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:48:22.862395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2faf0a-f890-4965-9ed5-f8ee079d3dff None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:22.941086 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10242dbf-c8a3-4e1d-bcfb-e3f29bb5e7a3 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:22.979448 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245a816-36b5-47e7-81e1-1d11cae305c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:22.980271 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245a816-36b5-47e7-81e1-1d11cae305c9 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:22.980550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245a816-36b5-47e7-81e1-1d11cae305c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:22.980830 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245a816-36b5-47e7-81e1-1d11cae305c9 None admin] PATH_INFO: `/v3/domains/default/users/2e3ec4696fd447c89f8d404bb4540c6c/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:22.983020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245a816-36b5-47e7-81e1-1d11cae305c9 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2e3ec4696fd447c89f8d404bb4540c6c, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:22.984502 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245a816-36b5-47e7-81e1-1d11cae305c9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:22.985382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2faf0a-f890-4965-9ed5-f8ee079d3dff None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:23.024281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02adddd0-63a2-4f35-a952-06eeac7b80b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:23.024962 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adddd0-63a2-4f35-a952-06eeac7b80b3 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:23.025902 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adddd0-63a2-4f35-a952-06eeac7b80b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:23.026394 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adddd0-63a2-4f35-a952-06eeac7b80b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:23.027444 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02adddd0-63a2-4f35-a952-06eeac7b80b3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:23.029037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02adddd0-63a2-4f35-a952-06eeac7b80b3 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:23.045098 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:23.070583 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 25/45] 10.222.0.27 () {58 vars in 1361 bytes} [Sat Jul 16 14:48:22 2022] PUT /identity/v3/domains/default/users/2e3ec4696fd447c89f8d404bb4540c6c/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 14:48:23.194756 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 21/46] 10.222.0.27 () {60 vars in 1141 bytes} [Sat Jul 16 14:48:22 2022] POST /identity/v3/projects => generated 312 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 14:48:25.171327 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356fb8f-0581-483c-9bdd-2bc4cf2cecea None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:25.171821 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356fb8f-0581-483c-9bdd-2bc4cf2cecea None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:25.171821 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356fb8f-0581-483c-9bdd-2bc4cf2cecea None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:25.173190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 26/47] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:25.178502 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca1656-eebd-4634-87f2-a30833e74451 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:25.178951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca1656-eebd-4634-87f2-a30833e74451 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:25.179111 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca1656-eebd-4634-87f2-a30833e74451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:25.196505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-0cca1656-eebd-4634-87f2-a30833e74451 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:25.197744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-0cca1656-eebd-4634-87f2-a30833e74451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:25.265492 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 22/48] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:25.288445 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356fb8f-0581-483c-9bdd-2bc4cf2cecea None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:25.306374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc79a0-dd22-4a2e-8132-5e8663ed2228 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:25.307300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc79a0-dd22-4a2e-8132-5e8663ed2228 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:25.307581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc79a0-dd22-4a2e-8132-5e8663ed2228 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:25.309110 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 23/49] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:48:25.314779 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c8925-5ec4-4ad5-9f87-60ba15b4b11e None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:25.315116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c8925-5ec4-4ad5-9f87-60ba15b4b11e None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:25.315390 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c8925-5ec4-4ad5-9f87-60ba15b4b11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:25.318672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:25.332149 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-da4c8925-5ec4-4ad5-9f87-60ba15b4b11e None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:25.339101 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-da4c8925-5ec4-4ad5-9f87-60ba15b4b11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:25.398276 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356fb8f-0581-483c-9bdd-2bc4cf2cecea None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:25.405926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 24/50] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 14:48:25.429497 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c8925-5ec4-4ad5-9f87-60ba15b4b11e None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:25.434619 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451b5ce-a2a6-4066-921d-b55fce40ce5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:25.435454 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451b5ce-a2a6-4066-921d-b55fce40ce5a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:25.435739 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451b5ce-a2a6-4066-921d-b55fce40ce5a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:25.436015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451b5ce-a2a6-4066-921d-b55fce40ce5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:25.446346 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451b5ce-a2a6-4066-921d-b55fce40ce5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:25.454642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-f451b5ce-a2a6-4066-921d-b55fce40ce5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:48:25.454642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:25.454642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:25.454642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:25.454642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:25.454642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:25.454642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:25.454642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:25.454642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:25.454642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:25.454642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:25.454642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:25.454642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:25.454642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:25.454642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:25.454642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:25.455658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:25.455658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:25.455658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:25.455658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:25.455658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:25.455658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:25.455658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:25.455658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:25.455658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:25.455658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:25.455658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:25.455658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:25.455658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:25.455658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:25.455658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:25.455658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:25.456396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:25.456396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:25.456396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:25.456396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:25.456396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:25.456396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:25.456396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:25.456396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:25.456396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:25.456396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:25.456396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:48:25.456396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:48:25.456396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 27/51] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:48:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:25.461818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451b5ce-a2a6-4066-921d-b55fce40ce5a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:25.487917 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:25.535541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4c8925-5ec4-4ad5-9f87-60ba15b4b11e None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:25.549418 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:25.566155 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451b5ce-a2a6-4066-921d-b55fce40ce5a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:25.574268 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56095d-2821-4cfb-8a66-8ba82eae6cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:25.575110 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56095d-2821-4cfb-8a66-8ba82eae6cda None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:25.575398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56095d-2821-4cfb-8a66-8ba82eae6cda None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:25.575677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56095d-2821-4cfb-8a66-8ba82eae6cda None admin] PATH_INFO: `/v3/domains/default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:25.576902 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56095d-2821-4cfb-8a66-8ba82eae6cda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:25.578387 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56095d-2821-4cfb-8a66-8ba82eae6cda None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:25.580162 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 25/52] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:48:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:48:25.585674 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56095d-2821-4cfb-8a66-8ba82eae6cda None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:25.602053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764c2e1-578a-415b-ab01-e3df2188fa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:25.602725 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764c2e1-578a-415b-ab01-e3df2188fa1d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:25.603035 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764c2e1-578a-415b-ab01-e3df2188fa1d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:25.604039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764c2e1-578a-415b-ab01-e3df2188fa1d None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:25.604925 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764c2e1-578a-415b-ab01-e3df2188fa1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:25.605233 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764c2e1-578a-415b-ab01-e3df2188fa1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:25.606623 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764c2e1-578a-415b-ab01-e3df2188fa1d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:25.615006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 28/53] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:48:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:48:25.621237 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764c2e1-578a-415b-ab01-e3df2188fa1d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:25.729798 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764c2e1-578a-415b-ab01-e3df2188fa1d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:25.767678 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc759a5a-d0e1-4b81-bba4-282b903592bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:25.768655 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc759a5a-d0e1-4b81-bba4-282b903592bc None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:25.768938 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc759a5a-d0e1-4b81-bba4-282b903592bc None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:25.769217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc759a5a-d0e1-4b81-bba4-282b903592bc None admin] PATH_INFO: `/v3/users/2e3ec4696fd447c89f8d404bb4540c6c` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:25.770476 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc759a5a-d0e1-4b81-bba4-282b903592bc None admin] RBAC: Authorizing `identity:get_user(user_id=2e3ec4696fd447c89f8d404bb4540c6c)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:25.772065 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc759a5a-d0e1-4b81-bba4-282b903592bc None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:25.778768 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 29/54] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:25 2022] GET /identity/v3/users/2e3ec4696fd447c89f8d404bb4540c6c => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:25.780171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc759a5a-d0e1-4b81-bba4-282b903592bc None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:25.798039 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56095d-2821-4cfb-8a66-8ba82eae6cda None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:25.799817 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:25.823361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:25.834017 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c99eaf-3023-46e3-afcb-a37d0098ba4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:25.834693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c99eaf-3023-46e3-afcb-a37d0098ba4c None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:25.835007 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c99eaf-3023-46e3-afcb-a37d0098ba4c None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:25.835292 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c99eaf-3023-46e3-afcb-a37d0098ba4c None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:25.836017 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c99eaf-3023-46e3-afcb-a37d0098ba4c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:25.838376 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c99eaf-3023-46e3-afcb-a37d0098ba4c None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:25.883932 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc759a5a-d0e1-4b81-bba4-282b903592bc None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:25.920890 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd3c85-b494-4b75-8fb1-3785f05b2985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:25.921692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd3c85-b494-4b75-8fb1-3785f05b2985 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:25.921980 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd3c85-b494-4b75-8fb1-3785f05b2985 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:25.922258 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd3c85-b494-4b75-8fb1-3785f05b2985 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:25.923570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd3c85-b494-4b75-8fb1-3785f05b2985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:25.925093 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd3c85-b494-4b75-8fb1-3785f05b2985 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:25.926742 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 30/55] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:48:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:48:25.932861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd3c85-b494-4b75-8fb1-3785f05b2985 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:26.017550 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 26/56] 10.222.0.27 () {60 vars in 1140 bytes} [Sat Jul 16 14:48:25 2022] POST /identity/v3/projects => generated 298 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 14:48:26.031422 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:26.039234 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd3c85-b494-4b75-8fb1-3785f05b2985 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:26.080623 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33473d06-4748-4497-aaf9-9e51b36d4dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:26.081305 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33473d06-4748-4497-aaf9-9e51b36d4dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:26.081593 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33473d06-4748-4497-aaf9-9e51b36d4dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:26.081871 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33473d06-4748-4497-aaf9-9e51b36d4dc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:26.082731 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33473d06-4748-4497-aaf9-9e51b36d4dc7 None admin] RBAC: Adding query filter params (role.id=0efeff46573247f3b9f1e2dec2968672, scope.domain.id=default, user.id=2e3ec4696fd447c89f8d404bb4540c6c) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:26.083109 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33473d06-4748-4497-aaf9-9e51b36d4dc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:26.084620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33473d06-4748-4497-aaf9-9e51b36d4dc7 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:26.092988 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 31/57] 10.222.0.27 () {56 vars in 1322 bytes} [Sat Jul 16 14:48:25 2022] GET /identity/v3/role_assignments?user.id=2e3ec4696fd447c89f8d404bb4540c6c&scope.domain.id=default&role.id=0efeff46573247f3b9f1e2dec2968672 => generated 530 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:48:28.095037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28c049-fbe3-42f8-859e-3b8491a14004 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:28.095567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28c049-fbe3-42f8-859e-3b8491a14004 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:28.095567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28c049-fbe3-42f8-859e-3b8491a14004 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:28.096939 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 27/58] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:28.101947 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8842cb-6c9f-4ea0-8331-75111cfba378 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:28.102321 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8842cb-6c9f-4ea0-8331-75111cfba378 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:28.102506 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8842cb-6c9f-4ea0-8331-75111cfba378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:28.120644 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3b8842cb-6c9f-4ea0-8331-75111cfba378 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:28.121838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3b8842cb-6c9f-4ea0-8331-75111cfba378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:28.188200 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 32/59] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:28.211643 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28c049-fbe3-42f8-859e-3b8491a14004 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:28.313613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28c049-fbe3-42f8-859e-3b8491a14004 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:28.346260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb046bf7-9c3c-40b4-8f9c-faccfca007d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:28.347068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb046bf7-9c3c-40b4-8f9c-faccfca007d0 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:28.347349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb046bf7-9c3c-40b4-8f9c-faccfca007d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:28.347620 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb046bf7-9c3c-40b4-8f9c-faccfca007d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:28.348781 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb046bf7-9c3c-40b4-8f9c-faccfca007d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:28.350217 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb046bf7-9c3c-40b4-8f9c-faccfca007d0 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:28.351855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 28/60] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:48:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:28.358056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8842cb-6c9f-4ea0-8331-75111cfba378 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:28.434954 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:28.456422 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8842cb-6c9f-4ea0-8331-75111cfba378 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:28.489366 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9e651-595a-4ddf-95e0-40c3c6dadd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:28.490055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9e651-595a-4ddf-95e0-40c3c6dadd57 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:28.490335 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9e651-595a-4ddf-95e0-40c3c6dadd57 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:28.490607 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9e651-595a-4ddf-95e0-40c3c6dadd57 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:28.491351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9e651-595a-4ddf-95e0-40c3c6dadd57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:28.492808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9e651-595a-4ddf-95e0-40c3c6dadd57 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:28.583428 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 33/61] 10.222.0.27 () {60 vars in 1132 bytes} [Sat Jul 16 14:48:28 2022] POST /identity/v3/users => generated 275 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:48:31.938520 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a670f8-4e3c-417f-95cb-5e2575de557d None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:31.940061 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a670f8-4e3c-417f-95cb-5e2575de557d None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:31.940799 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a670f8-4e3c-417f-95cb-5e2575de557d None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:31.943600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 29/62] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:31.950807 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d90978-d445-42b8-8505-f7055ca48297 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:31.952236 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d90978-d445-42b8-8505-f7055ca48297 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:31.953002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d90978-d445-42b8-8505-f7055ca48297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:31.997050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-61d90978-d445-42b8-8505-f7055ca48297 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:31.999120 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-61d90978-d445-42b8-8505-f7055ca48297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:32.106099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 34/63] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 14:48:32.145359 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a670f8-4e3c-417f-95cb-5e2575de557d None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:32.209311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:32.290604 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a670f8-4e3c-417f-95cb-5e2575de557d None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:32.342670 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac3a2c-ec0d-4f9a-97fb-a890b0ea7355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:32.343537 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac3a2c-ec0d-4f9a-97fb-a890b0ea7355 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:32.343825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac3a2c-ec0d-4f9a-97fb-a890b0ea7355 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:32.344106 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac3a2c-ec0d-4f9a-97fb-a890b0ea7355 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:32.360835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac3a2c-ec0d-4f9a-97fb-a890b0ea7355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:32.369076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-18ac3a2c-ec0d-4f9a-97fb-a890b0ea7355 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:48:32.369076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:32.369076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:32.369076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:32.369076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:32.369076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:32.369076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:32.369076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:32.369076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:32.369076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:32.369076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:32.369076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:32.369076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:32.369076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:32.369076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:32.369076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:32.370783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:32.370783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:32.370783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:32.370783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:32.370783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:32.370783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:32.370783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:32.370783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:32.370783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:32.370783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:32.370783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:32.370783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:32.370783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:32.370783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:32.370783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:32.370783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:32.373062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:32.373062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:32.373062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:32.373062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:32.373062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:32.373062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:32.373062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:32.373062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:32.373062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:32.373062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:32.373062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:48:32.373062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:48:32.373062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 30/64] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:48:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:32.382928 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d90978-d445-42b8-8505-f7055ca48297 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:32.472779 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:32.507971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81565715-0f95-4f2c-9b26-fb4af2ef8125 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:32.508378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81565715-0f95-4f2c-9b26-fb4af2ef8125 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:32.508606 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81565715-0f95-4f2c-9b26-fb4af2ef8125 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:32.510950 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 31/65] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:48:32.531540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d90978-d445-42b8-8505-f7055ca48297 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:32.534273 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9cb91-6ed8-4752-93cb-22ba1fef577f None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:32.534565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9cb91-6ed8-4752-93cb-22ba1fef577f None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:32.534850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9cb91-6ed8-4752-93cb-22ba1fef577f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:32.568167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-c2c9cb91-6ed8-4752-93cb-22ba1fef577f None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:32.569399 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c2c9cb91-6ed8-4752-93cb-22ba1fef577f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:32.570431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b878a-ee37-4322-9e98-ba2df9549a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:32.571145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b878a-ee37-4322-9e98-ba2df9549a84 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:32.571540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b878a-ee37-4322-9e98-ba2df9549a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:32.571867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b878a-ee37-4322-9e98-ba2df9549a84 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:32.573167 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b878a-ee37-4322-9e98-ba2df9549a84 None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:32.573521 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b878a-ee37-4322-9e98-ba2df9549a84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:32.575043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b878a-ee37-4322-9e98-ba2df9549a84 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:32.584492 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 35/66] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:48:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:32.585818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6baa6-93e4-4eef-a627-e54d94966614 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:32.586077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6baa6-93e4-4eef-a627-e54d94966614 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:32.586367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6baa6-93e4-4eef-a627-e54d94966614 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:32.588592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 36/67] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:32.592363 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6baa6-93e4-4eef-a627-e54d94966614 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:32.655428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 32/68] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 14:48:32.663934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b7a74-2869-4c07-b32f-42dd52463c32 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:32.664300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b7a74-2869-4c07-b32f-42dd52463c32 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:32.664540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b7a74-2869-4c07-b32f-42dd52463c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:32.698934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-1b3b7a74-2869-4c07-b32f-42dd52463c32 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:32.700246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-1b3b7a74-2869-4c07-b32f-42dd52463c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:32.709206 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6baa6-93e4-4eef-a627-e54d94966614 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:32.734600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:32.744421 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:32.761300 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6eb0e6-4ae7-49bb-a3fd-0cc727328f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:32.761300 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6eb0e6-4ae7-49bb-a3fd-0cc727328f6f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:32.761300 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6eb0e6-4ae7-49bb-a3fd-0cc727328f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:32.761300 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6eb0e6-4ae7-49bb-a3fd-0cc727328f6f None admin] PATH_INFO: `/v3/users/bcbfd29852d047909a1de71fb64876b1` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:32.775772 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6eb0e6-4ae7-49bb-a3fd-0cc727328f6f None admin] RBAC: Authorizing `identity:get_user(user_id=bcbfd29852d047909a1de71fb64876b1)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:32.778302 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6eb0e6-4ae7-49bb-a3fd-0cc727328f6f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:32.785653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 37/69] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:32 2022] GET /identity/v3/users/bcbfd29852d047909a1de71fb64876b1 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:48:32.785653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6eb0e6-4ae7-49bb-a3fd-0cc727328f6f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:32.793753 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 33/70] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:32.803602 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b7a74-2869-4c07-b32f-42dd52463c32 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:32.930936 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6eb0e6-4ae7-49bb-a3fd-0cc727328f6f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:32.967289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b7a74-2869-4c07-b32f-42dd52463c32 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:32.973593 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718442a-2f50-45f5-aef3-38440b62e1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:32.974390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718442a-2f50-45f5-aef3-38440b62e1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:32.974779 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718442a-2f50-45f5-aef3-38440b62e1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:32.975143 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718442a-2f50-45f5-aef3-38440b62e1b2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:32.988216 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718442a-2f50-45f5-aef3-38440b62e1b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:32.997370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-9718442a-2f50-45f5-aef3-38440b62e1b2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:48:32.997370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:32.997370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:32.997370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:32.997370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:32.997370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:32.997370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:32.997370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:32.997370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:32.997370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:32.997370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:32.997370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:32.997370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:32.997370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:32.997370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:32.997370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:32.999364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:32.999364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:32.999364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:32.999364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:32.999364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:32.999364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:32.999364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:32.999364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:32.999364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:32.999364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:32.999364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:32.999364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:32.999364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:32.999364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:32.999364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:32.999364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:33.000774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:33.000774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:33.000774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:33.000774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:33.000774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:33.000774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:33.000774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:33.000774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:33.000774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:33.000774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:33.000774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:48:33.000774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:48:33.000774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 38/71] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:48:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:33.000774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718442a-2f50-45f5-aef3-38440b62e1b2 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:33.017712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271a0c6-0115-4aaf-979a-2d583209207b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:33.018812 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271a0c6-0115-4aaf-979a-2d583209207b None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:33.019361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271a0c6-0115-4aaf-979a-2d583209207b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:33.019875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271a0c6-0115-4aaf-979a-2d583209207b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:33.057432 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:33.064125 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:33.113030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271a0c6-0115-4aaf-979a-2d583209207b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:33.114756 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271a0c6-0115-4aaf-979a-2d583209207b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:33.128033 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718442a-2f50-45f5-aef3-38440b62e1b2 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:33.173359 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a838e-e3e0-41d7-af07-fefab6f9e8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:33.174304 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a838e-e3e0-41d7-af07-fefab6f9e8ba None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:33.174705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a838e-e3e0-41d7-af07-fefab6f9e8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:33.175158 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a838e-e3e0-41d7-af07-fefab6f9e8ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:33.188645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a838e-e3e0-41d7-af07-fefab6f9e8ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:33.203398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-b05a838e-e3e0-41d7-af07-fefab6f9e8ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:48:33.203398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:33.203398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:33.203398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:33.203398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:33.203398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:33.203398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:33.203398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:33.203398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:33.203398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:33.203398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:33.203398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:33.203398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:33.203398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:33.203398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:33.203398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:33.204691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:33.204691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:33.204691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:33.204691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:33.204691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:33.204691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:33.204691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:33.204691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:33.204691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:33.204691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:33.204691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:33.204691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:33.204691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:33.204691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:33.204691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:33.204691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:33.205606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:33.205606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:33.205606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:33.205606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:33.205606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:33.205606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:33.205606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:33.205606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:33.205606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:33.205606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:33.205606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:48:33.205606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:48:33.205606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 39/72] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:48:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:33.206974 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e5dc5-b0e8-4ce3-9092-0e8b3b832451 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:33.207339 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e5dc5-b0e8-4ce3-9092-0e8b3b832451 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:33.207698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e5dc5-b0e8-4ce3-9092-0e8b3b832451 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:33.209589 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 40/73] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:33.210817 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e5dc5-b0e8-4ce3-9092-0e8b3b832451 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:33.259203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 34/74] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:48:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:48:33.261896 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d2c01-4b32-4c26-b8f0-e6399fdad4d1 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:33.262389 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d2c01-4b32-4c26-b8f0-e6399fdad4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:33.262881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d2c01-4b32-4c26-b8f0-e6399fdad4d1 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:33.265670 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 35/75] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:33.266797 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d2c01-4b32-4c26-b8f0-e6399fdad4d1 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:33.349288 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:33.354242 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e5dc5-b0e8-4ce3-9092-0e8b3b832451 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:33.357473 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:33.397416 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d2c01-4b32-4c26-b8f0-e6399fdad4d1 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:33.398449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f609e-d17f-4151-9694-6c13465729e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:33.399263 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f609e-d17f-4151-9694-6c13465729e8 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:33.399642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f609e-d17f-4151-9694-6c13465729e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:33.400072 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f609e-d17f-4151-9694-6c13465729e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:33.401084 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f609e-d17f-4151-9694-6c13465729e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:33.401577 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f609e-d17f-4151-9694-6c13465729e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:33.403542 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f609e-d17f-4151-9694-6c13465729e8 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:33.420078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 41/76] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:48:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:33.420078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73c136-0093-47b4-a8c9-f366e4648e76 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:33.420078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73c136-0093-47b4-a8c9-f366e4648e76 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:33.420078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73c136-0093-47b4-a8c9-f366e4648e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:33.437379 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5a73c136-0093-47b4-a8c9-f366e4648e76 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:33.438841 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5a73c136-0093-47b4-a8c9-f366e4648e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:33.449032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a621f2-d79e-4fa5-b2dc-d1bd59f20c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:33.449707 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a621f2-d79e-4fa5-b2dc-d1bd59f20c8a None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:33.449988 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a621f2-d79e-4fa5-b2dc-d1bd59f20c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:33.450262 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a621f2-d79e-4fa5-b2dc-d1bd59f20c8a None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:33.451067 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a621f2-d79e-4fa5-b2dc-d1bd59f20c8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:33.451394 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a621f2-d79e-4fa5-b2dc-d1bd59f20c8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:33.452813 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a621f2-d79e-4fa5-b2dc-d1bd59f20c8a None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:33.468147 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 36/77] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:48:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:33.469082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a621f2-d79e-4fa5-b2dc-d1bd59f20c8a None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:33.540573 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 42/78] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:33.542749 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad377c-ea39-40b6-8618-145d71ae5646 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:33.543290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad377c-ea39-40b6-8618-145d71ae5646 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:33.543766 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad377c-ea39-40b6-8618-145d71ae5646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:33.572829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-cdad377c-ea39-40b6-8618-145d71ae5646 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:33.574997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-cdad377c-ea39-40b6-8618-145d71ae5646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:33.614263 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a621f2-d79e-4fa5-b2dc-d1bd59f20c8a None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:33.646291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:33.656287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 43/79] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:33.657370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad377c-ea39-40b6-8618-145d71ae5646 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:33.661735 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032e996-25f3-4fe1-9dbf-e06c94dce5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:33.663313 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032e996-25f3-4fe1-9dbf-e06c94dce5bd None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:33.663848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032e996-25f3-4fe1-9dbf-e06c94dce5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:33.675876 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032e996-25f3-4fe1-9dbf-e06c94dce5bd None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:33.675876 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032e996-25f3-4fe1-9dbf-e06c94dce5bd None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:33.675876 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032e996-25f3-4fe1-9dbf-e06c94dce5bd None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:33.675876 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 37/80] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:48:33 2022] GET /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:33.675876 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032e996-25f3-4fe1-9dbf-e06c94dce5bd None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:33.691751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:33.774407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad377c-ea39-40b6-8618-145d71ae5646 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:33.820138 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472320b4-0da2-4e52-bb22-762d5f4cab90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:33.821430 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472320b4-0da2-4e52-bb22-762d5f4cab90 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:33.822416 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472320b4-0da2-4e52-bb22-762d5f4cab90 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:33.822927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472320b4-0da2-4e52-bb22-762d5f4cab90 None admin] PATH_INFO: `/v3/users/bcbfd29852d047909a1de71fb64876b1` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:33.824875 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472320b4-0da2-4e52-bb22-762d5f4cab90 None admin] RBAC: Authorizing `identity:get_user(user_id=bcbfd29852d047909a1de71fb64876b1)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:33.827336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472320b4-0da2-4e52-bb22-762d5f4cab90 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:33.829472 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032e996-25f3-4fe1-9dbf-e06c94dce5bd None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:33.830230 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 44/81] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:33 2022] GET /identity/v3/users/bcbfd29852d047909a1de71fb64876b1 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:33.834853 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472320b4-0da2-4e52-bb22-762d5f4cab90 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:33.886372 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44771d88-eeae-4568-b3ae-e9ea303e2985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:33.887508 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44771d88-eeae-4568-b3ae-e9ea303e2985 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:33.887913 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44771d88-eeae-4568-b3ae-e9ea303e2985 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:33.888303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44771d88-eeae-4568-b3ae-e9ea303e2985 None admin] PATH_INFO: `/v3/users/2e3ec4696fd447c89f8d404bb4540c6c` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:33.889854 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44771d88-eeae-4568-b3ae-e9ea303e2985 None admin] RBAC: Authorizing `identity:get_user(user_id=2e3ec4696fd447c89f8d404bb4540c6c)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:33.891548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44771d88-eeae-4568-b3ae-e9ea303e2985 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:33.893539 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 38/82] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:33 2022] GET /identity/v3/users/2e3ec4696fd447c89f8d404bb4540c6c => generated 247 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:33.894909 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44771d88-eeae-4568-b3ae-e9ea303e2985 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:33.969408 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:33.976829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472320b4-0da2-4e52-bb22-762d5f4cab90 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:33.997340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:34.024724 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383579f-37db-44de-9d6d-f0c46f549179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:34.027161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383579f-37db-44de-9d6d-f0c46f549179 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:34.027656 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383579f-37db-44de-9d6d-f0c46f549179 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:34.028187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383579f-37db-44de-9d6d-f0c46f549179 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:34.030160 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3383579f-37db-44de-9d6d-f0c46f549179 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:34.032859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3383579f-37db-44de-9d6d-f0c46f549179 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:34.036013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 45/83] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:48:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:34.037447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383579f-37db-44de-9d6d-f0c46f549179 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:34.048014 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44771d88-eeae-4568-b3ae-e9ea303e2985 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:34.086829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1d309-beeb-444d-8fb3-e9a4088b0a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:34.088542 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1d309-beeb-444d-8fb3-e9a4088b0a63 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:34.088907 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1d309-beeb-444d-8fb3-e9a4088b0a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:34.089266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc1d309-beeb-444d-8fb3-e9a4088b0a63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:34.090532 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1d309-beeb-444d-8fb3-e9a4088b0a63 None admin] RBAC: Adding query filter params (role.id=b7f92100b08f4dc6b2b6866a2955d2ab, scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532, user.id=bcbfd29852d047909a1de71fb64876b1) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:34.091077 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1d309-beeb-444d-8fb3-e9a4088b0a63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:34.092645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc1d309-beeb-444d-8fb3-e9a4088b0a63 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:34.101753 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 39/84] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:48:33 2022] GET /identity/v3/role_assignments?user.id=bcbfd29852d047909a1de71fb64876b1&scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532&role.id=b7f92100b08f4dc6b2b6866a2955d2ab => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:34.102624 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1d309-beeb-444d-8fb3-e9a4088b0a63 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:34.216198 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383579f-37db-44de-9d6d-f0c46f549179 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:34.257478 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc1d309-beeb-444d-8fb3-e9a4088b0a63 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:34.274373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e1eab1-295a-499f-b2c9-0f465c3ab3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:34.276344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1eab1-295a-499f-b2c9-0f465c3ab3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:34.276860 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1eab1-295a-499f-b2c9-0f465c3ab3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:34.277369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1eab1-295a-499f-b2c9-0f465c3ab3a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:34.294839 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e1eab1-295a-499f-b2c9-0f465c3ab3a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:34.300380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698e5d6-f35e-489a-b015-4a9222882eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:34.301038 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698e5d6-f35e-489a-b015-4a9222882eaa None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:34.301321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698e5d6-f35e-489a-b015-4a9222882eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:34.301596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698e5d6-f35e-489a-b015-4a9222882eaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:34.302518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-97e1eab1-295a-499f-b2c9-0f465c3ab3a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:48:34.302518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:34.302518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:34.302518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:34.302518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:34.302518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:34.302518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:34.302518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:34.302518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:34.302518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:34.302518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:34.302518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:34.302518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:34.302518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:34.302518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:34.302518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:34.304399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:34.304399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:34.304399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:34.304399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:34.304399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:34.304399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:34.304399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:34.304399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:34.304399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:34.304399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:34.304399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:34.304399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:34.304399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:34.304399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:34.304399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:34.304399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:34.305914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:34.305914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:34.305914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:34.305914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:34.305914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:34.305914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:34.305914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:34.305914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:34.305914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:34.305914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:34.305914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:48:34.305914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:48:34.305914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 46/85] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:48:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:34.305914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e1eab1-295a-499f-b2c9-0f465c3ab3a4 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:34.318650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:34.345881 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:34.413782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698e5d6-f35e-489a-b015-4a9222882eaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:34.417370 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698e5d6-f35e-489a-b015-4a9222882eaa None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:34.468384 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e1eab1-295a-499f-b2c9-0f465c3ab3a4 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:34.528417 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1a6bc-8713-48ed-af45-bf4f266bbfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:34.529469 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1a6bc-8713-48ed-af45-bf4f266bbfc6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:34.529987 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1a6bc-8713-48ed-af45-bf4f266bbfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:34.530476 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1a6bc-8713-48ed-af45-bf4f266bbfc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:34.551681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 40/86] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:48:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:48:34.552814 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698e5d6-f35e-489a-b015-4a9222882eaa None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:34.591920 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:34.637437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1a6bc-8713-48ed-af45-bf4f266bbfc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:34.639364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc1a6bc-8713-48ed-af45-bf4f266bbfc6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:34.668070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:34.703662 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698e5d6-f35e-489a-b015-4a9222882eaa None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:34.743339 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6dd72e-5ad8-4854-9fd1-b6c5ec17db53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:34.745023 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6dd72e-5ad8-4854-9fd1-b6c5ec17db53 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:34.745446 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6dd72e-5ad8-4854-9fd1-b6c5ec17db53 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:34.745727 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6dd72e-5ad8-4854-9fd1-b6c5ec17db53 None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:34.746462 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6dd72e-5ad8-4854-9fd1-b6c5ec17db53 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:34.747977 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6dd72e-5ad8-4854-9fd1-b6c5ec17db53 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:34.781348 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 47/87] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:48:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:48:34.782112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1a6bc-8713-48ed-af45-bf4f266bbfc6 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:34.847596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 41/88] 10.222.0.27 () {60 vars in 1132 bytes} [Sat Jul 16 14:48:34 2022] POST /identity/v3/users => generated 289 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:48:34.848464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6dd72e-5ad8-4854-9fd1-b6c5ec17db53 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:34.943634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1a6bc-8713-48ed-af45-bf4f266bbfc6 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:34.982415 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:34.991312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:35.001396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa39607-3d2f-4875-b4b1-02ba87ee9933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:35.002634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa39607-3d2f-4875-b4b1-02ba87ee9933 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:35.005340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa39607-3d2f-4875-b4b1-02ba87ee9933 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:35.005340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa39607-3d2f-4875-b4b1-02ba87ee9933 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:35.005340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa39607-3d2f-4875-b4b1-02ba87ee9933 None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:35.005340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa39607-3d2f-4875-b4b1-02ba87ee9933 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:35.007273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa39607-3d2f-4875-b4b1-02ba87ee9933 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:35.022952 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 48/89] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:48:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:35.022952 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa39607-3d2f-4875-b4b1-02ba87ee9933 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:35.025927 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6dd72e-5ad8-4854-9fd1-b6c5ec17db53 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:35.076216 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17594ce7-a576-4397-9fc0-e8d47284f6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:35.077110 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17594ce7-a576-4397-9fc0-e8d47284f6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:35.077397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17594ce7-a576-4397-9fc0-e8d47284f6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:35.077674 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17594ce7-a576-4397-9fc0-e8d47284f6d7 None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:35.078992 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17594ce7-a576-4397-9fc0-e8d47284f6d7 None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:35.080448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17594ce7-a576-4397-9fc0-e8d47284f6d7 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:35.082401 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 42/90] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:48:34 2022] GET /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:35.083041 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17594ce7-a576-4397-9fc0-e8d47284f6d7 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:35.180298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa39607-3d2f-4875-b4b1-02ba87ee9933 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:35.236219 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709372db-b836-41af-b356-9570fa945b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:35.237139 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709372db-b836-41af-b356-9570fa945b77 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:35.237415 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709372db-b836-41af-b356-9570fa945b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:35.237691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709372db-b836-41af-b356-9570fa945b77 None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:35.239299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709372db-b836-41af-b356-9570fa945b77 None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:35.241128 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709372db-b836-41af-b356-9570fa945b77 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:35.243189 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 49/91] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:48:35 2022] GET /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:35.244098 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709372db-b836-41af-b356-9570fa945b77 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:35.301154 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17594ce7-a576-4397-9fc0-e8d47284f6d7 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:35.350354 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:35.369953 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715dfdfc-957f-483c-9adc-c12930931cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:35.371547 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dfdfc-957f-483c-9adc-c12930931cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:35.372123 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dfdfc-957f-483c-9adc-c12930931cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:35.372984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dfdfc-957f-483c-9adc-c12930931cb8 None admin] PATH_INFO: `/v3/users/bcbfd29852d047909a1de71fb64876b1` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:35.375022 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715dfdfc-957f-483c-9adc-c12930931cb8 None admin] RBAC: Authorizing `identity:get_user(user_id=bcbfd29852d047909a1de71fb64876b1)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:35.377253 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715dfdfc-957f-483c-9adc-c12930931cb8 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:35.379306 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 43/92] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:35 2022] GET /identity/v3/users/bcbfd29852d047909a1de71fb64876b1 => generated 275 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:35.379851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715dfdfc-957f-483c-9adc-c12930931cb8 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:35.415446 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:35.452543 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709372db-b836-41af-b356-9570fa945b77 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:35.519962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf917ea-c5bf-485d-b15f-bc43093e300b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:35.521112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf917ea-c5bf-485d-b15f-bc43093e300b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:35.521614 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf917ea-c5bf-485d-b15f-bc43093e300b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:35.522126 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf917ea-c5bf-485d-b15f-bc43093e300b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:35.523684 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf917ea-c5bf-485d-b15f-bc43093e300b None admin] RBAC: Adding query filter params (role.id=af11cd5265d14ac7902b1fb3afcc4be7, scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532, user.id=bcbfd29852d047909a1de71fb64876b1) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:35.524260 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf917ea-c5bf-485d-b15f-bc43093e300b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:35.527195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf917ea-c5bf-485d-b15f-bc43093e300b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:35.538509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 50/93] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:48:35 2022] GET /identity/v3/role_assignments?user.id=bcbfd29852d047909a1de71fb64876b1&scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532&role.id=af11cd5265d14ac7902b1fb3afcc4be7 => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:35.544509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf917ea-c5bf-485d-b15f-bc43093e300b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:35.629163 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715dfdfc-957f-483c-9adc-c12930931cb8 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:35.749269 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21a33e-d338-4cd4-aedb-478e7b081f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:35.750253 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21a33e-d338-4cd4-aedb-478e7b081f00 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:35.750557 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21a33e-d338-4cd4-aedb-478e7b081f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:35.751105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21a33e-d338-4cd4-aedb-478e7b081f00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:35.752791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21a33e-d338-4cd4-aedb-478e7b081f00 None admin] RBAC: Adding query filter params (role.id=0efeff46573247f3b9f1e2dec2968672, scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532, user.id=2e3ec4696fd447c89f8d404bb4540c6c) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:35.753363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21a33e-d338-4cd4-aedb-478e7b081f00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:35.755954 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21a33e-d338-4cd4-aedb-478e7b081f00 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:35.787696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 44/94] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:48:35 2022] GET /identity/v3/role_assignments?user.id=2e3ec4696fd447c89f8d404bb4540c6c&scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532&role.id=0efeff46573247f3b9f1e2dec2968672 => generated 259 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:35.930155 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf917ea-c5bf-485d-b15f-bc43093e300b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:35.958172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:35.995517 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e87e4-5cf7-4390-90b5-8d0e6a9bfa59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:35.997076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e87e4-5cf7-4390-90b5-8d0e6a9bfa59 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:35.997867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e87e4-5cf7-4390-90b5-8d0e6a9bfa59 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:35.998850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e87e4-5cf7-4390-90b5-8d0e6a9bfa59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:36.161660 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e87e4-5cf7-4390-90b5-8d0e6a9bfa59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:36.165209 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e87e4-5cf7-4390-90b5-8d0e6a9bfa59 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:36.400960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 51/95] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:48:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 862 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:48:36.409469 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21a33e-d338-4cd4-aedb-478e7b081f00 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:36.540197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:36.562752 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21a33e-d338-4cd4-aedb-478e7b081f00 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:36.623172 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9196a459-0fd8-4b43-a077-6d1771676114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:36.624062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196a459-0fd8-4b43-a077-6d1771676114 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:36.624342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196a459-0fd8-4b43-a077-6d1771676114 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:36.624613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9196a459-0fd8-4b43-a077-6d1771676114 None admin] PATH_INFO: `/v3/projects/7f20b07ad9f648969064ba9000eb5a8e` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:36.631829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196a459-0fd8-4b43-a077-6d1771676114 None admin] RBAC: Authorizing `identity:get_project(project_id=7f20b07ad9f648969064ba9000eb5a8e)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:36.633336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9196a459-0fd8-4b43-a077-6d1771676114 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:36.635781 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 45/96] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:48:36 2022] GET /identity/v3/projects/7f20b07ad9f648969064ba9000eb5a8e => generated 312 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:36.645293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e87e4-5cf7-4390-90b5-8d0e6a9bfa59 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:36.705446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:36.883362 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e87e4-5cf7-4390-90b5-8d0e6a9bfa59 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:36.961644 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9565fbee-98ca-4ac0-a5c8-d217ef09c03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:36.963562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565fbee-98ca-4ac0-a5c8-d217ef09c03a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:36.964048 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565fbee-98ca-4ac0-a5c8-d217ef09c03a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:36.964430 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9565fbee-98ca-4ac0-a5c8-d217ef09c03a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:36.965323 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9565fbee-98ca-4ac0-a5c8-d217ef09c03a None admin] RBAC: Adding query filter params (role.id=b7f92100b08f4dc6b2b6866a2955d2ab, scope.project.id=7f20b07ad9f648969064ba9000eb5a8e, user.id=bcbfd29852d047909a1de71fb64876b1) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:36.965674 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9565fbee-98ca-4ac0-a5c8-d217ef09c03a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:36.967437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9565fbee-98ca-4ac0-a5c8-d217ef09c03a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:36.976396 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3bd09-ab90-4784-bdc7-3090400216a0 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:36.976693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3bd09-ab90-4784-bdc7-3090400216a0 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:36.976952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3bd09-ab90-4784-bdc7-3090400216a0 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:36.983153 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 52/97] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:48:36 2022] GET /identity/v3/role_assignments?user.id=bcbfd29852d047909a1de71fb64876b1&scope.project.id=7f20b07ad9f648969064ba9000eb5a8e&role.id=b7f92100b08f4dc6b2b6866a2955d2ab => generated 259 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:36.983421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 46/98] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:36.991215 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42ae0f-a7a9-404e-a320-4703d7938a7c None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:36.991714 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42ae0f-a7a9-404e-a320-4703d7938a7c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:36.992208 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42ae0f-a7a9-404e-a320-4703d7938a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:37.024573 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-cf42ae0f-a7a9-404e-a320-4703d7938a7c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:37.027592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-cf42ae0f-a7a9-404e-a320-4703d7938a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:37.178980 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 53/99] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:37.210323 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec3bd09-ab90-4784-bdc7-3090400216a0 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:37.417751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec3bd09-ab90-4784-bdc7-3090400216a0 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:37.485052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8cfb4-7753-4f87-a5b8-5fc15a0a8cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:37.486395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8cfb4-7753-4f87-a5b8-5fc15a0a8cbe None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:37.486939 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8cfb4-7753-4f87-a5b8-5fc15a0a8cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:37.487421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8cfb4-7753-4f87-a5b8-5fc15a0a8cbe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:37.503897 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e8cfb4-7753-4f87-a5b8-5fc15a0a8cbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:37.513361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-a4e8cfb4-7753-4f87-a5b8-5fc15a0a8cbe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:48:37.513361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:37.513361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:37.513361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:37.513361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:37.513361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:37.513361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:37.513361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:37.513361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:37.513361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:37.513361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:37.513361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:37.513361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:37.513361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:37.513361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:37.513361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:37.514298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:37.514298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:37.514298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:37.514298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:37.514298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:37.514298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:37.514298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:37.514298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:37.514298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:37.514298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:37.514298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:37.514298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:37.514298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:37.514298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:37.514298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:37.514298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:37.515068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:37.515068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:37.515068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:37.515068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:37.515068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:37.515068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:37.515068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:37.515068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:37.515068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:37.515068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:37.515068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:48:37.515068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:48:37.518153 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 47/100] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:48:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:37.524833 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42ae0f-a7a9-404e-a320-4703d7938a7c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:37.542500 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:37.663974 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf42ae0f-a7a9-404e-a320-4703d7938a7c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:37.720585 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8af8f3-edeb-433a-900e-955f3bc0ffd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:37.721297 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8af8f3-edeb-433a-900e-955f3bc0ffd1 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:37.721586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8af8f3-edeb-433a-900e-955f3bc0ffd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:37.721868 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8af8f3-edeb-433a-900e-955f3bc0ffd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:37.722646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8af8f3-edeb-433a-900e-955f3bc0ffd1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:37.722981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8af8f3-edeb-433a-900e-955f3bc0ffd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:37.724685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8af8f3-edeb-433a-900e-955f3bc0ffd1 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:37.742640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 54/101] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:48:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:48:37.744974 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8cfb4-7753-4f87-a5b8-5fc15a0a8cbe None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:37.785918 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:37.985643 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8cfb4-7753-4f87-a5b8-5fc15a0a8cbe None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:38.048223 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddcdfe6-fdcc-4aef-95ea-b05181b13da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:38.049651 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcdfe6-fdcc-4aef-95ea-b05181b13da0 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:38.050159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcdfe6-fdcc-4aef-95ea-b05181b13da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:38.050655 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcdfe6-fdcc-4aef-95ea-b05181b13da0 None admin] PATH_INFO: `/v3/users/bcbfd29852d047909a1de71fb64876b1` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:38.052890 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddcdfe6-fdcc-4aef-95ea-b05181b13da0 None admin] RBAC: Authorizing `identity:get_user(user_id=bcbfd29852d047909a1de71fb64876b1)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:38.055719 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddcdfe6-fdcc-4aef-95ea-b05181b13da0 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:38.059429 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 48/102] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:37 2022] GET /identity/v3/users/bcbfd29852d047909a1de71fb64876b1 => generated 275 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:38.065395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8af8f3-edeb-433a-900e-955f3bc0ffd1 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:38.207068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:38.259197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8af8f3-edeb-433a-900e-955f3bc0ffd1 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:38.297772 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04c700-e3d4-463b-8926-1e3ba411c3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:38.298716 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04c700-e3d4-463b-8926-1e3ba411c3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:38.298968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04c700-e3d4-463b-8926-1e3ba411c3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:38.299251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04c700-e3d4-463b-8926-1e3ba411c3e9 None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:38.300499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04c700-e3d4-463b-8926-1e3ba411c3e9 None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:38.301981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04c700-e3d4-463b-8926-1e3ba411c3e9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:38.304086 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 55/103] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:48:38 2022] GET /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:38.310529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddcdfe6-fdcc-4aef-95ea-b05181b13da0 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:38.428204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:38.460556 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddcdfe6-fdcc-4aef-95ea-b05181b13da0 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:38.511820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a519032-d65b-4fc9-bea1-a0553614569d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:38.512836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a519032-d65b-4fc9-bea1-a0553614569d None admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:38.513128 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a519032-d65b-4fc9-bea1-a0553614569d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:38.513408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a519032-d65b-4fc9-bea1-a0553614569d None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532/users/bcbfd29852d047909a1de71fb64876b1/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:38.515876 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a519032-d65b-4fc9-bea1-a0553614569d None admin] RBAC: Authorizing `identity:create_grant(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532, user_id=bcbfd29852d047909a1de71fb64876b1, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:38.517400 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a519032-d65b-4fc9-bea1-a0553614569d None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:38.609277 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 49/104] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:48:38 2022] PUT /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532/users/bcbfd29852d047909a1de71fb64876b1/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:48:39.022390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5a95f-668b-42c2-b592-a900e187e0a3 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:39.022959 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5a95f-668b-42c2-b592-a900e187e0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:39.023465 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5a95f-668b-42c2-b592-a900e187e0a3 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:39.025639 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 56/105] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:39.031931 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82fcfc5-6155-49d8-b63b-f5834e366074 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:39.032222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82fcfc5-6155-49d8-b63b-f5834e366074 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:39.032501 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82fcfc5-6155-49d8-b63b-f5834e366074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:39.068890 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-b82fcfc5-6155-49d8-b63b-f5834e366074 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:39.071043 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b82fcfc5-6155-49d8-b63b-f5834e366074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:39.160316 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22630883-9983-4ddc-b0c7-1a552f289c21 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:39.161299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22630883-9983-4ddc-b0c7-1a552f289c21 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:39.161797 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22630883-9983-4ddc-b0c7-1a552f289c21 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:39.163562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 57/106] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:39.170848 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7a6cb-d54d-4dc3-81e9-b9c9aeb7359e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:39.171501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7a6cb-d54d-4dc3-81e9-b9c9aeb7359e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:39.172027 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7a6cb-d54d-4dc3-81e9-b9c9aeb7359e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:39.216056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-66a7a6cb-d54d-4dc3-81e9-b9c9aeb7359e None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:39.217749 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-66a7a6cb-d54d-4dc3-81e9-b9c9aeb7359e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:39.241651 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 50/107] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 14:48:39.243246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a19f5-c0fc-409e-bde3-6984bff508bd None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:39.244073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a19f5-c0fc-409e-bde3-6984bff508bd None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:39.244361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a19f5-c0fc-409e-bde3-6984bff508bd None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:39.245712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 51/108] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:39.247051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad190b0-9dda-429d-be2c-be72b4f887f6 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:39.247339 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad190b0-9dda-429d-be2c-be72b4f887f6 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:39.247619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad190b0-9dda-429d-be2c-be72b4f887f6 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:39.249122 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 52/109] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:39.254936 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376a5c1-7887-443b-bc4d-45232e1620c2 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:39.255229 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376a5c1-7887-443b-bc4d-45232e1620c2 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:39.255510 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376a5c1-7887-443b-bc4d-45232e1620c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:39.282327 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-2376a5c1-7887-443b-bc4d-45232e1620c2 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:39.283622 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-2376a5c1-7887-443b-bc4d-45232e1620c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:39.334760 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 58/110] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 14:48:39.336333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d42c3-a86c-4237-bdca-0f5d6d86ed66 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:39.336630 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d42c3-a86c-4237-bdca-0f5d6d86ed66 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:39.336913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d42c3-a86c-4237-bdca-0f5d6d86ed66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:39.355704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-447d42c3-a86c-4237-bdca-0f5d6d86ed66 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:39.357013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-447d42c3-a86c-4237-bdca-0f5d6d86ed66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:39.362789 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:39.370418 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 53/111] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:39.371608 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376a5c1-7887-443b-bc4d-45232e1620c2 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:39.457358 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:39.463445 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 59/112] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:39.464358 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d42c3-a86c-4237-bdca-0f5d6d86ed66 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:39.494593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376a5c1-7887-443b-bc4d-45232e1620c2 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:39.536042 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129772e-cceb-422f-b804-11ffaee812ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:39.537162 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129772e-cceb-422f-b804-11ffaee812ef None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:39.537588 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129772e-cceb-422f-b804-11ffaee812ef None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:39.538001 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129772e-cceb-422f-b804-11ffaee812ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:39.539540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129772e-cceb-422f-b804-11ffaee812ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:39.541702 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129772e-cceb-422f-b804-11ffaee812ef None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:39.543758 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 54/113] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:48:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:39.544844 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129772e-cceb-422f-b804-11ffaee812ef None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:39.587487 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d42c3-a86c-4237-bdca-0f5d6d86ed66 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:39.628752 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db2de4-81ef-43cd-9169-3bd3ed6f59f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:39.628967 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:39.629059 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db2de4-81ef-43cd-9169-3bd3ed6f59f4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:39.629228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db2de4-81ef-43cd-9169-3bd3ed6f59f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:39.629513 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db2de4-81ef-43cd-9169-3bd3ed6f59f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:39.642524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db2de4-81ef-43cd-9169-3bd3ed6f59f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:39.651849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-91db2de4-81ef-43cd-9169-3bd3ed6f59f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:48:39.651849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:39.651849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:39.651849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:39.651849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:39.651849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:39.651849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:39.651849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:39.651849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:39.651849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:39.651849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:39.651849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:39.651849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:39.651849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:39.651849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:39.651849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:39.652940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:39.652940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:39.652940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:39.652940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:39.652940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:39.652940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:39.652940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:39.652940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:39.652940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:39.652940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:39.652940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:39.652940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:39.652940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:39.652940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:39.652940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:39.652940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:39.653770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:39.653770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:39.653770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:39.653770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:39.653770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:39.653770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:39.653770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:39.653770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:39.653770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:39.653770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:39.653770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:48:39.653770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:48:39.653770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 60/114] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:48:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:39.654601 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db2de4-81ef-43cd-9169-3bd3ed6f59f4 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:39.668591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129772e-cceb-422f-b804-11ffaee812ef None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:39.708123 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47966d33-785a-4505-8fca-2c5b0bae74c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:39.709158 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47966d33-785a-4505-8fca-2c5b0bae74c5 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:39.709582 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47966d33-785a-4505-8fca-2c5b0bae74c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:39.710066 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47966d33-785a-4505-8fca-2c5b0bae74c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:39.723371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47966d33-785a-4505-8fca-2c5b0bae74c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:39.732054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-47966d33-785a-4505-8fca-2c5b0bae74c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:48:39.732054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:39.732054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:39.732054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:39.732054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:39.732054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:39.732054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:39.732054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:39.732054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:39.732054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:39.732054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:39.732054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:39.732054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:39.732054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:39.732054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:39.732054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:39.732938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:39.732938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:39.732938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:39.732938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:39.732938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:39.732938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:39.732938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:39.732938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:39.732938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:39.732938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:39.732938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:39.732938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:39.732938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:39.732938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:39.732938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:39.732938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:39.733677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:39.733677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:39.733677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:39.733677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:39.733677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:39.733677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:39.733677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:39.733677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:39.733677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:39.733677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:39.733677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:48:39.733677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:48:39.734242 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:39.736615 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 55/115] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:48:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:39.737678 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47966d33-785a-4505-8fca-2c5b0bae74c5 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:39.785326 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db2de4-81ef-43cd-9169-3bd3ed6f59f4 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:39.826287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb56c6-894c-4de6-a43e-a4a13afae5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:39.827347 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb56c6-894c-4de6-a43e-a4a13afae5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:39.827551 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb56c6-894c-4de6-a43e-a4a13afae5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:39.827875 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb56c6-894c-4de6-a43e-a4a13afae5b6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:39.841775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb56c6-894c-4de6-a43e-a4a13afae5b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:39.850728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-2ecb56c6-894c-4de6-a43e-a4a13afae5b6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 14:48:39.850728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:39.850728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:39.850728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:39.850728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:39.850728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:39.850728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:39.850728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:39.850728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:39.850728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:39.850728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:39.850728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:39.850728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:39.850728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:39.850728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:39.850728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:39.851687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:39.851687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:39.851687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:39.851687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:39.851687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:39.851687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:39.851687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:39.851687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:39.851687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:39.851687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:39.851687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:39.851687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:39.851687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:39.851687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:39.851687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:39.851687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:39.852431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:39.852431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:39.852431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:39.852431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:39.852431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:39.852431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:39.852431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:39.852431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:39.852431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:39.852431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:39.852431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 14:48:39.852431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:48:39.852431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 61/116] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:48:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:39.858177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47966d33-785a-4505-8fca-2c5b0bae74c5 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:39.858533 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb56c6-894c-4de6-a43e-a4a13afae5b6 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:39.894034 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:39.907997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea761c-6798-4319-903c-84830fe00591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:39.911691 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea761c-6798-4319-903c-84830fe00591 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:39.911691 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea761c-6798-4319-903c-84830fe00591 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:39.911691 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea761c-6798-4319-903c-84830fe00591 None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:39.911691 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea761c-6798-4319-903c-84830fe00591 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:39.913025 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ea761c-6798-4319-903c-84830fe00591 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:39.981524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb56c6-894c-4de6-a43e-a4a13afae5b6 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:40.004913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:40.020464 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e84a6b-e40e-43dc-b1dc-c231fb34792e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:40.020936 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e84a6b-e40e-43dc-b1dc-c231fb34792e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:40.021232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e84a6b-e40e-43dc-b1dc-c231fb34792e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:40.021514 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e84a6b-e40e-43dc-b1dc-c231fb34792e None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:40.022283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e84a6b-e40e-43dc-b1dc-c231fb34792e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:40.022599 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e84a6b-e40e-43dc-b1dc-c231fb34792e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:40.024151 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e84a6b-e40e-43dc-b1dc-c231fb34792e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:40.029642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 56/117] 10.222.0.27 () {60 vars in 1141 bytes} [Sat Jul 16 14:48:39 2022] POST /identity/v3/projects => generated 302 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:48:40.030858 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea761c-6798-4319-903c-84830fe00591 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:40.036045 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 62/118] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:48:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:40.037888 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e84a6b-e40e-43dc-b1dc-c231fb34792e None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:40.180397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea761c-6798-4319-903c-84830fe00591 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:40.232039 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fd108-b922-4443-8cba-20865e1ea980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:40.233093 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fd108-b922-4443-8cba-20865e1ea980 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:40.233620 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fd108-b922-4443-8cba-20865e1ea980 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:40.234093 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fd108-b922-4443-8cba-20865e1ea980 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:40.235128 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3fd108-b922-4443-8cba-20865e1ea980 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:40.236017 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3fd108-b922-4443-8cba-20865e1ea980 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:40.238257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3fd108-b922-4443-8cba-20865e1ea980 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:40.242120 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e84a6b-e40e-43dc-b1dc-c231fb34792e None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:40.252140 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 57/119] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:48:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:40.253383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fd108-b922-4443-8cba-20865e1ea980 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:40.310510 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88581b1-f829-416f-9850-0e2817c3dce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:40.310851 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88581b1-f829-416f-9850-0e2817c3dce9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:40.311250 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88581b1-f829-416f-9850-0e2817c3dce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:40.311527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88581b1-f829-416f-9850-0e2817c3dce9 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:40.312301 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88581b1-f829-416f-9850-0e2817c3dce9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:40.312608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88581b1-f829-416f-9850-0e2817c3dce9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:40.313994 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:40.314104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88581b1-f829-416f-9850-0e2817c3dce9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:40.325212 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 63/120] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:48:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:40.328013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88581b1-f829-416f-9850-0e2817c3dce9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:40.397199 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:40.414187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fd108-b922-4443-8cba-20865e1ea980 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:40.465736 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e6e62-b9ea-49cb-b54e-7521a49bb3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:40.467026 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e6e62-b9ea-49cb-b54e-7521a49bb3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:40.467327 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e6e62-b9ea-49cb-b54e-7521a49bb3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:40.467614 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e6e62-b9ea-49cb-b54e-7521a49bb3f0 None admin] PATH_INFO: `/v3/users/bcbfd29852d047909a1de71fb64876b1` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:40.469724 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e6e62-b9ea-49cb-b54e-7521a49bb3f0 None admin] RBAC: Authorizing `identity:get_user(user_id=bcbfd29852d047909a1de71fb64876b1)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:40.471565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e6e62-b9ea-49cb-b54e-7521a49bb3f0 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:40.472473 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88581b1-f829-416f-9850-0e2817c3dce9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:40.473655 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 58/121] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:40 2022] GET /identity/v3/users/bcbfd29852d047909a1de71fb64876b1 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:40.478990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e6e62-b9ea-49cb-b54e-7521a49bb3f0 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:40.519483 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc7995-a077-438d-85c0-2f22d87356c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:40.520651 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc7995-a077-438d-85c0-2f22d87356c3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:40.521097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc7995-a077-438d-85c0-2f22d87356c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:40.521513 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc7995-a077-438d-85c0-2f22d87356c3 None admin] PATH_INFO: `/v3/users/2e3ec4696fd447c89f8d404bb4540c6c` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:40.523551 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc7995-a077-438d-85c0-2f22d87356c3 None admin] RBAC: Authorizing `identity:get_user(user_id=2e3ec4696fd447c89f8d404bb4540c6c)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:40.526092 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc7995-a077-438d-85c0-2f22d87356c3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:40.528307 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 64/122] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:40 2022] GET /identity/v3/users/2e3ec4696fd447c89f8d404bb4540c6c => generated 247 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:40.529986 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a480b-9f2f-41da-9973-a17ab6987288 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:40.530460 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a480b-9f2f-41da-9973-a17ab6987288 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:40.530921 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a480b-9f2f-41da-9973-a17ab6987288 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:40.532469 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 65/123] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:40.533680 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a480b-9f2f-41da-9973-a17ab6987288 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:40.620831 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e6e62-b9ea-49cb-b54e-7521a49bb3f0 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:40.640241 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:40.664186 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a480b-9f2f-41da-9973-a17ab6987288 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:40.668031 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cff338-177d-4331-bb51-aeefa84d0c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:40.669926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cff338-177d-4331-bb51-aeefa84d0c72 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:40.670648 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cff338-177d-4331-bb51-aeefa84d0c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:40.671243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cff338-177d-4331-bb51-aeefa84d0c72 None admin] PATH_INFO: `/v3/users/2f686cad97234a329e2f1f46b2f6258b` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:40.683992 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:40.692736 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cff338-177d-4331-bb51-aeefa84d0c72 None admin] RBAC: Authorizing `identity:get_user(user_id=2f686cad97234a329e2f1f46b2f6258b)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:40.694589 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cff338-177d-4331-bb51-aeefa84d0c72 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:40.696711 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 59/124] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:40 2022] GET /identity/v3/users/2f686cad97234a329e2f1f46b2f6258b => generated 289 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:40.703181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cff338-177d-4331-bb51-aeefa84d0c72 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:40.711555 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c4cef-1019-4057-9da5-a3d8e6da5590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:40.713636 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c4cef-1019-4057-9da5-a3d8e6da5590 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:40.714176 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c4cef-1019-4057-9da5-a3d8e6da5590 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:40.714817 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c4cef-1019-4057-9da5-a3d8e6da5590 None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:40.716576 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c4cef-1019-4057-9da5-a3d8e6da5590 None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:40.719207 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c4cef-1019-4057-9da5-a3d8e6da5590 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:40.721759 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 66/125] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:48:40 2022] GET /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:40.723408 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476b63c-7a99-4e30-bf40-ed131eac9aae None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:40.723833 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476b63c-7a99-4e30-bf40-ed131eac9aae None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:40.724261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476b63c-7a99-4e30-bf40-ed131eac9aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:40.750371 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a476b63c-7a99-4e30-bf40-ed131eac9aae None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:40.751856 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a476b63c-7a99-4e30-bf40-ed131eac9aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:40.861491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 67/126] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:40.867050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cff338-177d-4331-bb51-aeefa84d0c72 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:40.869189 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476b63c-7a99-4e30-bf40-ed131eac9aae None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:40.922778 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c468657-efbe-41c7-bbbb-5e70eacdf076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:40.924279 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c468657-efbe-41c7-bbbb-5e70eacdf076 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:40.924770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c468657-efbe-41c7-bbbb-5e70eacdf076 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:40.925233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c468657-efbe-41c7-bbbb-5e70eacdf076 None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:40.926849 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c468657-efbe-41c7-bbbb-5e70eacdf076 None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:40.928925 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c468657-efbe-41c7-bbbb-5e70eacdf076 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:40.931106 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 60/127] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:48:40 2022] GET /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:40.932554 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c468657-efbe-41c7-bbbb-5e70eacdf076 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:40.973838 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:41.011479 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476b63c-7a99-4e30-bf40-ed131eac9aae None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:41.030662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:41.054881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c468657-efbe-41c7-bbbb-5e70eacdf076 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:41.075334 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92770b-e315-4667-8248-006abd858526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:41.076395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92770b-e315-4667-8248-006abd858526 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:41.076907 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92770b-e315-4667-8248-006abd858526 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:41.077508 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92770b-e315-4667-8248-006abd858526 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:41.100182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fbde8-579d-420a-aa9b-1d797d098906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:41.101524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fbde8-579d-420a-aa9b-1d797d098906 None admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:41.102052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fbde8-579d-420a-aa9b-1d797d098906 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:41.102620 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fbde8-579d-420a-aa9b-1d797d098906 None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532/users/bcbfd29852d047909a1de71fb64876b1/roles/af11cd5265d14ac7902b1fb3afcc4be7` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:41.106616 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fbde8-579d-420a-aa9b-1d797d098906 None admin] RBAC: Authorizing `identity:create_grant(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532, user_id=bcbfd29852d047909a1de71fb64876b1, role_id=af11cd5265d14ac7902b1fb3afcc4be7)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:41.108830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fbde8-579d-420a-aa9b-1d797d098906 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:41.217584 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 61/128] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:48:40 2022] PUT /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532/users/bcbfd29852d047909a1de71fb64876b1/roles/af11cd5265d14ac7902b1fb3afcc4be7 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:48:41.218979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fbde8-579d-420a-aa9b-1d797d098906 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:41.219508 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92770b-e315-4667-8248-006abd858526 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:41.222722 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92770b-e315-4667-8248-006abd858526 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:41.369125 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:41.379801 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fbde8-579d-420a-aa9b-1d797d098906 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:41.430574 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25284cf-74a7-4db7-ad35-9b415fe3d290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:41.433267 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25284cf-74a7-4db7-ad35-9b415fe3d290 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:41.433944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25284cf-74a7-4db7-ad35-9b415fe3d290 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:41.436433 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25284cf-74a7-4db7-ad35-9b415fe3d290 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:41.451372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 68/129] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:48:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:48:41.452853 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25284cf-74a7-4db7-ad35-9b415fe3d290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:41.453060 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92770b-e315-4667-8248-006abd858526 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:41.461349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-e25284cf-74a7-4db7-ad35-9b415fe3d290 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:48:41.461349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:41.461349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:41.461349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:41.461349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:41.461349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:41.461349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:41.461349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:41.461349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:41.461349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:41.461349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:41.461349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:41.461349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:41.461349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:41.461349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:41.461349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:41.463249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:41.463249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:41.463249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:41.463249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:41.463249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:41.463249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:41.463249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:41.463249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:41.463249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:41.463249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:41.463249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:41.463249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:41.463249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:41.463249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:41.463249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:41.463249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:41.464209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:41.464209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:41.464209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:41.464209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:41.464209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:41.464209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:41.464209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:41.464209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:41.464209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:41.464209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:41.464209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:48:41.464209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:48:41.466951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 62/130] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:48:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:41.468796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e0243-a723-403d-9c06-d0f99a9e57b9 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:41.469367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e0243-a723-403d-9c06-d0f99a9e57b9 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:41.469855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e0243-a723-403d-9c06-d0f99a9e57b9 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:41.471711 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 63/131] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:41.473104 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e0243-a723-403d-9c06-d0f99a9e57b9 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:41.491368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:41.624205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e0243-a723-403d-9c06-d0f99a9e57b9 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:41.636704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92770b-e315-4667-8248-006abd858526 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:41.673682 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc05ac4-a54e-4abe-8291-be317697e042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:41.674828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc05ac4-a54e-4abe-8291-be317697e042 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:41.675211 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc05ac4-a54e-4abe-8291-be317697e042 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:41.675562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc05ac4-a54e-4abe-8291-be317697e042 None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:41.677572 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc05ac4-a54e-4abe-8291-be317697e042 None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:41.679332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc05ac4-a54e-4abe-8291-be317697e042 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:41.681718 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 64/132] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:48:41 2022] GET /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:41.684112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976336d-3d71-45a0-a77d-776afc61f957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:41.685173 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976336d-3d71-45a0-a77d-776afc61f957 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:41.685519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976336d-3d71-45a0-a77d-776afc61f957 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:41.685873 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976336d-3d71-45a0-a77d-776afc61f957 None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532/users/2e3ec4696fd447c89f8d404bb4540c6c/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:41.688177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09d91e-c9ad-4b6c-a50c-b127f572d46f None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:41.688306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976336d-3d71-45a0-a77d-776afc61f957 None admin] RBAC: Authorizing `identity:create_grant(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532, user_id=2e3ec4696fd447c89f8d404bb4540c6c, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:41.688557 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09d91e-c9ad-4b6c-a50c-b127f572d46f None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:41.688847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09d91e-c9ad-4b6c-a50c-b127f572d46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:41.690431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976336d-3d71-45a0-a77d-776afc61f957 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:41.709203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-da09d91e-c9ad-4b6c-a50c-b127f572d46f None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:41.710449 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-da09d91e-c9ad-4b6c-a50c-b127f572d46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:41.731706 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:41.761962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 69/133] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:48:41 2022] PUT /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532/users/2e3ec4696fd447c89f8d404bb4540c6c/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:48:41.763398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976336d-3d71-45a0-a77d-776afc61f957 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:41.817063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 65/134] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:41.818014 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09d91e-c9ad-4b6c-a50c-b127f572d46f None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:41.869218 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:41.928744 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976336d-3d71-45a0-a77d-776afc61f957 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:41.977151 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e3f67-f42d-4c7c-9150-38663fd20c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:41.978911 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e3f67-f42d-4c7c-9150-38663fd20c8a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:41.979704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e3f67-f42d-4c7c-9150-38663fd20c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:41.980848 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e3f67-f42d-4c7c-9150-38663fd20c8a None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:41.982030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e3f67-f42d-4c7c-9150-38663fd20c8a None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:41.982576 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e3f67-f42d-4c7c-9150-38663fd20c8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:41.984631 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e3f67-f42d-4c7c-9150-38663fd20c8a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:41.997423 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 70/135] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:48:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:41.998660 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e3f67-f42d-4c7c-9150-38663fd20c8a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:42.066023 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09d91e-c9ad-4b6c-a50c-b127f572d46f None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:42.112229 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da33a16-8041-42f9-a867-5bb119501a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:42.112901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da33a16-8041-42f9-a867-5bb119501a9f None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:42.113187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da33a16-8041-42f9-a867-5bb119501a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:42.113457 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da33a16-8041-42f9-a867-5bb119501a9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:42.114315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da33a16-8041-42f9-a867-5bb119501a9f None admin] RBAC: Adding query filter params (role.id=7f966f669d82448daddfe911e2955ea2, scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532, user.id=2f686cad97234a329e2f1f46b2f6258b) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:42.114631 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da33a16-8041-42f9-a867-5bb119501a9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:42.116223 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da33a16-8041-42f9-a867-5bb119501a9f None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:42.124703 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 66/136] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:48:41 2022] GET /identity/v3/role_assignments?user.id=2f686cad97234a329e2f1f46b2f6258b&scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532&role.id=7f966f669d82448daddfe911e2955ea2 => generated 259 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:42.125587 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da33a16-8041-42f9-a867-5bb119501a9f None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:42.149021 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e3f67-f42d-4c7c-9150-38663fd20c8a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:42.182864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:42.194007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:42.201344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3267e-284e-415b-bd44-13dc00ab261c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:42.202778 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3267e-284e-415b-bd44-13dc00ab261c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:42.203404 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3267e-284e-415b-bd44-13dc00ab261c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:42.203601 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3267e-284e-415b-bd44-13dc00ab261c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:42.248249 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3267e-284e-415b-bd44-13dc00ab261c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:42.256369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-f2b3267e-284e-415b-bd44-13dc00ab261c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:48:42.256369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:42.256369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:42.256369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:42.256369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:42.256369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:42.256369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:42.256369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:42.256369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:42.256369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:42.256369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:42.256369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:42.256369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:42.256369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:42.256369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:42.256369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:42.257266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:42.257266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:42.257266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:42.257266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:42.257266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:42.257266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:42.257266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:42.257266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:42.257266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:42.257266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:42.257266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:42.257266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:42.257266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:42.257266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:42.257266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:42.257266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:42.258006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:42.258006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:42.258006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:42.258006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:42.258006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:42.258006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:42.258006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:42.258006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:42.258006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:42.258006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:42.258006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:48:42.258006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:48:42.264732 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 71/137] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:48:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:42.270819 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3267e-284e-415b-bd44-13dc00ab261c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:42.315367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da33a16-8041-42f9-a867-5bb119501a9f None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:42.361152 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1710e5-9df9-40f5-bed9-ab4464b87d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:42.362474 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1710e5-9df9-40f5-bed9-ab4464b87d54 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:42.363010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1710e5-9df9-40f5-bed9-ab4464b87d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:42.363495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1710e5-9df9-40f5-bed9-ab4464b87d54 None admin] PATH_INFO: `/v3/users/bcbfd29852d047909a1de71fb64876b1` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:42.365853 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1710e5-9df9-40f5-bed9-ab4464b87d54 None admin] RBAC: Authorizing `identity:get_user(user_id=bcbfd29852d047909a1de71fb64876b1)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:42.367936 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1710e5-9df9-40f5-bed9-ab4464b87d54 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:42.370491 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 67/138] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:42 2022] GET /identity/v3/users/bcbfd29852d047909a1de71fb64876b1 => generated 275 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:42.379304 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1710e5-9df9-40f5-bed9-ab4464b87d54 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:42.481859 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:42.501093 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1710e5-9df9-40f5-bed9-ab4464b87d54 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:42.529416 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3267e-284e-415b-bd44-13dc00ab261c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:42.543916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fc5cd-8a8a-458f-8c99-f131fae5298f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:42.545167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fc5cd-8a8a-458f-8c99-f131fae5298f None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:42.545646 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fc5cd-8a8a-458f-8c99-f131fae5298f None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:42.546129 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fc5cd-8a8a-458f-8c99-f131fae5298f None admin] PATH_INFO: `/v3/projects/7f20b07ad9f648969064ba9000eb5a8e` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:42.547696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4fc5cd-8a8a-458f-8c99-f131fae5298f None admin] RBAC: Authorizing `identity:get_project(project_id=7f20b07ad9f648969064ba9000eb5a8e)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:42.549417 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4fc5cd-8a8a-458f-8c99-f131fae5298f None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:42.551523 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 68/139] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:48:42 2022] GET /identity/v3/projects/7f20b07ad9f648969064ba9000eb5a8e => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:48:42.559969 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fc5cd-8a8a-458f-8c99-f131fae5298f None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:42.618134 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b1cbe-aba0-4c1f-ab7e-baf28c62718f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:42.620307 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b1cbe-aba0-4c1f-ab7e-baf28c62718f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:42.621957 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b1cbe-aba0-4c1f-ab7e-baf28c62718f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:42.623075 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b1cbe-aba0-4c1f-ab7e-baf28c62718f None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:42.624968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b1cbe-aba0-4c1f-ab7e-baf28c62718f None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:42.626992 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b1cbe-aba0-4c1f-ab7e-baf28c62718f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:42.628690 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b1cbe-aba0-4c1f-ab7e-baf28c62718f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:42.655488 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 72/140] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:48:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:48:42.661687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b1cbe-aba0-4c1f-ab7e-baf28c62718f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:42.776098 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:42.833570 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fc5cd-8a8a-458f-8c99-f131fae5298f None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:42.885152 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b1cbe-aba0-4c1f-ab7e-baf28c62718f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:42.937877 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f80a4f-5949-41c9-b6cc-4fac8f6e4cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:42.940177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f80a4f-5949-41c9-b6cc-4fac8f6e4cde None admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:42.943697 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f80a4f-5949-41c9-b6cc-4fac8f6e4cde None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:42.944222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f80a4f-5949-41c9-b6cc-4fac8f6e4cde None admin] PATH_INFO: `/v3/projects/7f20b07ad9f648969064ba9000eb5a8e/users/bcbfd29852d047909a1de71fb64876b1/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:42.955949 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f80a4f-5949-41c9-b6cc-4fac8f6e4cde None admin] RBAC: Authorizing `identity:create_grant(project_id=7f20b07ad9f648969064ba9000eb5a8e, user_id=bcbfd29852d047909a1de71fb64876b1, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:42.957811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f80a4f-5949-41c9-b6cc-4fac8f6e4cde None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:42.985759 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6641a57-3caa-417c-b44a-510a93524df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:42.995130 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6641a57-3caa-417c-b44a-510a93524df9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:42.995130 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6641a57-3caa-417c-b44a-510a93524df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:42.995130 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6641a57-3caa-417c-b44a-510a93524df9 None admin] PATH_INFO: `/v3/users/bcbfd29852d047909a1de71fb64876b1` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:42.995130 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6641a57-3caa-417c-b44a-510a93524df9 None admin] RBAC: Authorizing `identity:get_user(user_id=bcbfd29852d047909a1de71fb64876b1)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:42.995130 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6641a57-3caa-417c-b44a-510a93524df9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:42.995130 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 73/141] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:42 2022] GET /identity/v3/users/bcbfd29852d047909a1de71fb64876b1 => generated 275 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:48:42.998540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6641a57-3caa-417c-b44a-510a93524df9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:43.078374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 69/142] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:48:42 2022] PUT /identity/v3/projects/7f20b07ad9f648969064ba9000eb5a8e/users/bcbfd29852d047909a1de71fb64876b1/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:48:43.179104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6641a57-3caa-417c-b44a-510a93524df9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:43.216237 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:43.229452 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248043d-3550-42b9-ae0e-4540cb5d3e38 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:43.233104 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248043d-3550-42b9-ae0e-4540cb5d3e38 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:43.233104 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248043d-3550-42b9-ae0e-4540cb5d3e38 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:43.233104 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 70/143] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:48:43.233617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74827629-2105-4ec0-812e-4277cfb829ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:43.234191 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74827629-2105-4ec0-812e-4277cfb829ed None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:43.234466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74827629-2105-4ec0-812e-4277cfb829ed None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:43.234744 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74827629-2105-4ec0-812e-4277cfb829ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:43.237248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb760ee-2fc9-4402-81a8-296225efa606 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:43.237584 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb760ee-2fc9-4402-81a8-296225efa606 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:43.237867 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb760ee-2fc9-4402-81a8-296225efa606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:43.272462 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-4fb760ee-2fc9-4402-81a8-296225efa606 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:43.273691 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-4fb760ee-2fc9-4402-81a8-296225efa606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:43.295378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:43.357520 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74827629-2105-4ec0-812e-4277cfb829ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:43.359080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74827629-2105-4ec0-812e-4277cfb829ed None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:43.477132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 74/144] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:48:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:48:43.483107 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74827629-2105-4ec0-812e-4277cfb829ed None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:43.511509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:43.512527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 71/145] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 14:48:43.545451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb760ee-2fc9-4402-81a8-296225efa606 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:43.605037 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74827629-2105-4ec0-812e-4277cfb829ed None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:43.669421 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4033812-6378-4704-be06-1874cb12746b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:43.670588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4033812-6378-4704-be06-1874cb12746b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:43.671016 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4033812-6378-4704-be06-1874cb12746b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:43.671403 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4033812-6378-4704-be06-1874cb12746b None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:43.673285 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4033812-6378-4704-be06-1874cb12746b None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:43.675627 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4033812-6378-4704-be06-1874cb12746b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:43.679287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 75/146] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:48:43 2022] GET /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:43.689394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4033812-6378-4704-be06-1874cb12746b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:43.825175 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb760ee-2fc9-4402-81a8-296225efa606 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:43.910601 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:43.930656 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4033812-6378-4704-be06-1874cb12746b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:43.939060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30282b15-584e-4839-af98-a4d93b94cc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:43.939915 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30282b15-584e-4839-af98-a4d93b94cc9f None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:43.940202 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30282b15-584e-4839-af98-a4d93b94cc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:43.940481 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30282b15-584e-4839-af98-a4d93b94cc9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:43.942844 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30282b15-584e-4839-af98-a4d93b94cc9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:43.944698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30282b15-584e-4839-af98-a4d93b94cc9f None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:43.951435 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 72/147] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:48:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:48:43.952979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30282b15-584e-4839-af98-a4d93b94cc9f None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:43.987320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:44.007168 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17022eb2-0cb7-46fe-90b5-1872a8d33231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:44.008207 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17022eb2-0cb7-46fe-90b5-1872a8d33231 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:44.008635 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17022eb2-0cb7-46fe-90b5-1872a8d33231 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:44.009620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17022eb2-0cb7-46fe-90b5-1872a8d33231 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:44.011636 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17022eb2-0cb7-46fe-90b5-1872a8d33231 None admin] RBAC: Adding query filter params (role.id=b7f92100b08f4dc6b2b6866a2955d2ab, scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532, user.id=bcbfd29852d047909a1de71fb64876b1) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:44.012150 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17022eb2-0cb7-46fe-90b5-1872a8d33231 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:44.014904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17022eb2-0cb7-46fe-90b5-1872a8d33231 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:44.028068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 76/148] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:48:43 2022] GET /identity/v3/role_assignments?user.id=bcbfd29852d047909a1de71fb64876b1&scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532&role.id=b7f92100b08f4dc6b2b6866a2955d2ab => generated 608 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:44.182828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30282b15-584e-4839-af98-a4d93b94cc9f None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:44.258511 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e94573-386e-4c08-9300-5cb639a5e280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:44.260016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e94573-386e-4c08-9300-5cb639a5e280 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:44.260532 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e94573-386e-4c08-9300-5cb639a5e280 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:44.261049 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e94573-386e-4c08-9300-5cb639a5e280 None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:44.262448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e94573-386e-4c08-9300-5cb639a5e280 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:44.265355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e94573-386e-4c08-9300-5cb639a5e280 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:44.370625 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadc42e-c7d5-48f3-a8a8-7bbcfc1c6bab None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:44.371201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadc42e-c7d5-48f3-a8a8-7bbcfc1c6bab None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:44.371692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadc42e-c7d5-48f3-a8a8-7bbcfc1c6bab None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:44.374801 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 77/149] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:44.381106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504dff9d-6a74-4782-affa-5db0eb76144d None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:44.381590 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504dff9d-6a74-4782-affa-5db0eb76144d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:44.382078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504dff9d-6a74-4782-affa-5db0eb76144d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:44.405866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 73/150] 10.222.0.27 () {60 vars in 1132 bytes} [Sat Jul 16 14:48:43 2022] POST /identity/v3/users => generated 283 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:48:44.425073 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-504dff9d-6a74-4782-affa-5db0eb76144d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:44.427481 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-504dff9d-6a74-4782-affa-5db0eb76144d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:44.586440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 78/151] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 14:48:44.614766 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e94573-386e-4c08-9300-5cb639a5e280 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:44.771381 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:44.849348 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e94573-386e-4c08-9300-5cb639a5e280 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:44.918025 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec1e5ef-998f-46bd-bb53-60c6500426fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:44.919029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2711b-d491-4973-a625-8b439ad3e529 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:44.919626 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1e5ef-998f-46bd-bb53-60c6500426fd None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:44.920065 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2711b-d491-4973-a625-8b439ad3e529 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:44.920165 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1e5ef-998f-46bd-bb53-60c6500426fd None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:44.920656 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1e5ef-998f-46bd-bb53-60c6500426fd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:44.920756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2711b-d491-4973-a625-8b439ad3e529 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:44.922440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 79/152] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:48:44.929338 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e80f6-3f7c-49ab-9005-764164adae0e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:44.929951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e80f6-3f7c-49ab-9005-764164adae0e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:44.930453 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e80f6-3f7c-49ab-9005-764164adae0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:44.945548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec1e5ef-998f-46bd-bb53-60c6500426fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:44.954935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-dec1e5ef-998f-46bd-bb53-60c6500426fd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:48:44.954935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:44.954935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:44.954935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:44.954935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:44.954935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:44.954935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:44.954935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:44.954935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:44.954935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:44.954935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:44.954935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:44.954935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:44.954935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:44.954935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:44.954935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:44.955856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:44.955856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:44.955856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:44.955856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:44.955856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:44.955856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:44.955856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:44.955856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:44.955856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:44.955856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:44.955856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:44.955856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:44.955856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:44.955856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:44.955856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:44.955856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:44.956601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:44.956601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:44.956601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:44.956601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:44.956601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:44.956601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:44.956601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:44.956601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:44.956601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:44.956601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:44.956601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:48:44.956601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:48:44.959341 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 74/153] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:48:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:44.968534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-6d6e80f6-3f7c-49ab-9005-764164adae0e None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:44.968750 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec1e5ef-998f-46bd-bb53-60c6500426fd None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:44.969455 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6d6e80f6-3f7c-49ab-9005-764164adae0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:45.079846 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 80/154] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:45.109821 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e80f6-3f7c-49ab-9005-764164adae0e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:45.149973 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:45.192813 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec1e5ef-998f-46bd-bb53-60c6500426fd None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:45.257768 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:45.282463 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990a472-3011-4dc5-ac10-5fe09677b0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:45.283603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990a472-3011-4dc5-ac10-5fe09677b0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:45.284073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990a472-3011-4dc5-ac10-5fe09677b0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:45.284534 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990a472-3011-4dc5-ac10-5fe09677b0e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:45.285894 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990a472-3011-4dc5-ac10-5fe09677b0e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:45.286405 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990a472-3011-4dc5-ac10-5fe09677b0e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:45.288890 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990a472-3011-4dc5-ac10-5fe09677b0e5 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:45.317875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 75/155] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:48:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:45.328696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990a472-3011-4dc5-ac10-5fe09677b0e5 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:45.432409 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e80f6-3f7c-49ab-9005-764164adae0e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:45.518363 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693678dc-5aa6-4f98-a0c0-825264fcbd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:45.519229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693678dc-5aa6-4f98-a0c0-825264fcbd65 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:45.519567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693678dc-5aa6-4f98-a0c0-825264fcbd65 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:45.519911 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693678dc-5aa6-4f98-a0c0-825264fcbd65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:45.547515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693678dc-5aa6-4f98-a0c0-825264fcbd65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:45.552809 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990a472-3011-4dc5-ac10-5fe09677b0e5 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:45.564246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-693678dc-5aa6-4f98-a0c0-825264fcbd65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:48:45.564246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:45.564246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:45.564246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:45.564246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:45.564246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:45.564246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:45.564246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:45.564246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:45.564246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:45.564246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:45.564246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:45.564246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:45.564246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:45.564246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:45.564246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:45.565228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:45.565228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:45.565228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:45.565228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:45.565228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:45.565228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:45.565228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:45.565228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:45.565228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:45.565228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:45.565228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:45.565228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:45.565228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:45.565228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:45.565228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:45.565228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:45.565978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:45.565978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:45.565978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:45.565978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:45.565978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:45.565978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:45.565978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:45.565978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:45.565978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:45.565978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:45.565978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:48:45.565978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:48:45.566557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 81/156] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:48:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 457 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:48:45.591776 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693678dc-5aa6-4f98-a0c0-825264fcbd65 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:45.641970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd6e6f-e2ba-446e-a37d-aced9a7f7c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:45.641970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd6e6f-e2ba-446e-a37d-aced9a7f7c68 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:45.641970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd6e6f-e2ba-446e-a37d-aced9a7f7c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:45.641970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd6e6f-e2ba-446e-a37d-aced9a7f7c68 None admin] PATH_INFO: `/v3/users/bcbfd29852d047909a1de71fb64876b1` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:45.643402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd6e6f-e2ba-446e-a37d-aced9a7f7c68 None admin] RBAC: Authorizing `identity:get_user(user_id=bcbfd29852d047909a1de71fb64876b1)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:45.645092 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd6e6f-e2ba-446e-a37d-aced9a7f7c68 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:45.648458 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 76/157] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:45 2022] GET /identity/v3/users/bcbfd29852d047909a1de71fb64876b1 => generated 275 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:45.655786 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd6e6f-e2ba-446e-a37d-aced9a7f7c68 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:45.795363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:45.827387 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:45.854532 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd6e6f-e2ba-446e-a37d-aced9a7f7c68 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:45.911129 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3131a89-6058-492b-abe5-c32d3e0baea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:45.912150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3131a89-6058-492b-abe5-c32d3e0baea3 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:45.912615 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3131a89-6058-492b-abe5-c32d3e0baea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:45.913031 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3131a89-6058-492b-abe5-c32d3e0baea3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:45.928114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693678dc-5aa6-4f98-a0c0-825264fcbd65 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:46.002872 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ddf94-a0b2-4eb9-aa05-b156c5a7ae13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:46.003820 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ddf94-a0b2-4eb9-aa05-b156c5a7ae13 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:46.004306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ddf94-a0b2-4eb9-aa05-b156c5a7ae13 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:46.004716 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ddf94-a0b2-4eb9-aa05-b156c5a7ae13 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:46.005612 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ddf94-a0b2-4eb9-aa05-b156c5a7ae13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:46.007728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ddf94-a0b2-4eb9-aa05-b156c5a7ae13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:46.009652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ddf94-a0b2-4eb9-aa05-b156c5a7ae13 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:46.026454 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 82/158] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:48:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:48:46.030933 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc83bcf-9b9a-4a1f-aeb3-112c7d9cdcc5 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:46.032387 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc83bcf-9b9a-4a1f-aeb3-112c7d9cdcc5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:46.035662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc83bcf-9b9a-4a1f-aeb3-112c7d9cdcc5 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:46.038704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 83/159] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:46 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:46.041859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc83bcf-9b9a-4a1f-aeb3-112c7d9cdcc5 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:46.073951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3131a89-6058-492b-abe5-c32d3e0baea3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:46.076055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3131a89-6058-492b-abe5-c32d3e0baea3 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:46.187940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:46.275763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 77/160] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:48:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 16 14:48:46.278258 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701e21c-dd2d-44a0-9f2a-9443a916002c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:46.278941 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701e21c-dd2d-44a0-9f2a-9443a916002c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:46.280744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701e21c-dd2d-44a0-9f2a-9443a916002c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:46.294117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc83bcf-9b9a-4a1f-aeb3-112c7d9cdcc5 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:46.318210 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-f701e21c-dd2d-44a0-9f2a-9443a916002c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:46.321908 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-f701e21c-dd2d-44a0-9f2a-9443a916002c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:46.362774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda99dab-0d53-410b-b285-ff9d13989af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:46.365136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda99dab-0d53-410b-b285-ff9d13989af4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:46.367569 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda99dab-0d53-410b-b285-ff9d13989af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:46.368047 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda99dab-0d53-410b-b285-ff9d13989af4 None admin] PATH_INFO: `/v3/users/2e3ec4696fd447c89f8d404bb4540c6c` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:46.372401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda99dab-0d53-410b-b285-ff9d13989af4 None admin] RBAC: Authorizing `identity:get_user(user_id=2e3ec4696fd447c89f8d404bb4540c6c)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:46.379251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda99dab-0d53-410b-b285-ff9d13989af4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:46.379413 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 84/161] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:46 2022] GET /identity/v3/users/2e3ec4696fd447c89f8d404bb4540c6c => generated 247 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:46.382749 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda99dab-0d53-410b-b285-ff9d13989af4 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:46.388195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:46.529465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 78/162] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:46.532033 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadab02d-f24a-4da8-a913-9dfb24db9e65 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:46.532727 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadab02d-f24a-4da8-a913-9dfb24db9e65 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:46.538923 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadab02d-f24a-4da8-a913-9dfb24db9e65 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:46.542479 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 79/163] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:46 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:46.550920 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadab02d-f24a-4da8-a913-9dfb24db9e65 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:46.604706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda99dab-0d53-410b-b285-ff9d13989af4 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:46.674244 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7def8-df9b-452b-9097-57e72c7a169f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:46.676704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7def8-df9b-452b-9097-57e72c7a169f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:46.677185 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7def8-df9b-452b-9097-57e72c7a169f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:46.677655 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7def8-df9b-452b-9097-57e72c7a169f None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:46.679794 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7def8-df9b-452b-9097-57e72c7a169f None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:46.683685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7def8-df9b-452b-9097-57e72c7a169f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:46.690934 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 85/164] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:48:46 2022] GET /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532 => generated 298 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:46.690934 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747ef44-d80e-4124-b61d-0c0f96d80ced None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:46.690934 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747ef44-d80e-4124-b61d-0c0f96d80ced None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:46.691373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747ef44-d80e-4124-b61d-0c0f96d80ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:46.749326 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b747ef44-d80e-4124-b61d-0c0f96d80ced None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:46.751357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b747ef44-d80e-4124-b61d-0c0f96d80ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:46.852639 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:46.905972 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadab02d-f24a-4da8-a913-9dfb24db9e65 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:46.943973 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 86/165] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:46.946936 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747ef44-d80e-4124-b61d-0c0f96d80ced None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:46.952034 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:47.001184 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2330e-6401-4b90-b0a2-8bf55039bb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:47.002659 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2330e-6401-4b90-b0a2-8bf55039bb15 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:47.003448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2330e-6401-4b90-b0a2-8bf55039bb15 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:47.006482 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2330e-6401-4b90-b0a2-8bf55039bb15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:47.183524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac2330e-6401-4b90-b0a2-8bf55039bb15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:47.187901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac2330e-6401-4b90-b0a2-8bf55039bb15 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:47.318207 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747ef44-d80e-4124-b61d-0c0f96d80ced None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:47.377288 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:47.411401 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 80/166] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:48:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 868 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:48:47.415102 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2330e-6401-4b90-b0a2-8bf55039bb15 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:47.479734 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73acf6-5af1-4afb-96ca-39b823c757cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:47.481091 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73acf6-5af1-4afb-96ca-39b823c757cf None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:47.481575 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73acf6-5af1-4afb-96ca-39b823c757cf None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:47.482067 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73acf6-5af1-4afb-96ca-39b823c757cf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:47.528857 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e73acf6-5af1-4afb-96ca-39b823c757cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:47.560447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-5e73acf6-5af1-4afb-96ca-39b823c757cf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 14:48:47.560447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:47.560447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:47.560447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:47.560447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:47.560447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:47.560447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:47.560447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:47.560447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:47.560447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:47.560447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:47.560447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:47.560447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:47.560447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:47.560447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:47.560447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:47.562174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:47.562174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:47.562174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:47.562174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:47.562174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:47.562174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:47.562174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:47.562174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:47.562174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:47.562174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:47.562174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:47.562174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:47.562174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:47.562174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:47.562174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:47.562174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:47.563977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:47.563977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:47.563977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:47.563977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:47.563977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:47.563977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:47.563977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:47.563977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:47.563977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:47.563977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:47.563977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 14:48:47.563977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:48:47.563977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 87/167] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:48:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 617 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:47.565227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73acf6-5af1-4afb-96ca-39b823c757cf None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:47.635325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2330e-6401-4b90-b0a2-8bf55039bb15 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:47.674315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8481b4b-0d11-4b6b-9624-7bf9c61addc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:47.675307 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8481b4b-0d11-4b6b-9624-7bf9c61addc6 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:47.675733 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8481b4b-0d11-4b6b-9624-7bf9c61addc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:47.676148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8481b4b-0d11-4b6b-9624-7bf9c61addc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:47.676366 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:47.677301 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8481b4b-0d11-4b6b-9624-7bf9c61addc6 None admin] RBAC: Adding query filter params (role.id=af11cd5265d14ac7902b1fb3afcc4be7, scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532, user.id=bcbfd29852d047909a1de71fb64876b1) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:47.677824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8481b4b-0d11-4b6b-9624-7bf9c61addc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:47.679562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8481b4b-0d11-4b6b-9624-7bf9c61addc6 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:47.686522 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 81/168] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:48:47 2022] GET /identity/v3/role_assignments?user.id=bcbfd29852d047909a1de71fb64876b1&scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532&role.id=af11cd5265d14ac7902b1fb3afcc4be7 => generated 608 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:47.687718 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8481b4b-0d11-4b6b-9624-7bf9c61addc6 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:47.723691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73acf6-5af1-4afb-96ca-39b823c757cf None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:47.745721 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:47.771591 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45620248-121c-4a23-9ca9-50a037f93d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:47.772499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45620248-121c-4a23-9ca9-50a037f93d86 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:47.772849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45620248-121c-4a23-9ca9-50a037f93d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:47.773189 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45620248-121c-4a23-9ca9-50a037f93d86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:47.796478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45620248-121c-4a23-9ca9-50a037f93d86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:47.808638 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8481b4b-0d11-4b6b-9624-7bf9c61addc6 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:47.812633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-45620248-121c-4a23-9ca9-50a037f93d86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:48:47.812633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:47.812633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:47.812633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:47.812633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:47.812633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:47.812633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:47.812633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:47.812633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:47.812633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:47.812633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:47.812633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:47.812633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:47.812633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:47.812633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:47.812633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:47.813588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:47.813588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:47.813588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:47.813588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:47.813588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:47.813588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:47.813588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:47.813588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:47.813588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:47.813588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:47.813588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:47.813588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:47.813588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:47.813588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:47.813588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:47.813588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:47.814399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:47.814399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:47.814399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:47.814399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:47.814399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:47.814399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:47.814399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:47.814399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:47.814399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:47.814399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:47.814399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:48:47.814399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:48:47.814399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 88/169] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:48:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:47.815085 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45620248-121c-4a23-9ca9-50a037f93d86 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:47.852389 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736048ea-bcb9-4f2b-9b94-e7d588772343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:47.853613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736048ea-bcb9-4f2b-9b94-e7d588772343 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:47.854181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736048ea-bcb9-4f2b-9b94-e7d588772343 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:47.854635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736048ea-bcb9-4f2b-9b94-e7d588772343 None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:47.856351 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736048ea-bcb9-4f2b-9b94-e7d588772343 None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:47.858034 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736048ea-bcb9-4f2b-9b94-e7d588772343 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:47.860434 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 82/170] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:48:47 2022] GET /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:47.862259 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d1e06-af46-42da-9b59-fda1788c9439 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:47.862799 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d1e06-af46-42da-9b59-fda1788c9439 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:47.863411 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d1e06-af46-42da-9b59-fda1788c9439 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:47.865086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 83/171] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:47.866091 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d1e06-af46-42da-9b59-fda1788c9439 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:48.024010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d1e06-af46-42da-9b59-fda1788c9439 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:48.046710 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:48.074192 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15936fcf-75a2-48cf-9b6a-c9f2399c33e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:48.075358 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15936fcf-75a2-48cf-9b6a-c9f2399c33e7 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:48.075844 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15936fcf-75a2-48cf-9b6a-c9f2399c33e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:48.076518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15936fcf-75a2-48cf-9b6a-c9f2399c33e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:48.077762 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15936fcf-75a2-48cf-9b6a-c9f2399c33e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:48.078307 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15936fcf-75a2-48cf-9b6a-c9f2399c33e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:48.081061 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15936fcf-75a2-48cf-9b6a-c9f2399c33e7 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:48.089545 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45620248-121c-4a23-9ca9-50a037f93d86 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:48.094155 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 84/172] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:48:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:48.095659 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15936fcf-75a2-48cf-9b6a-c9f2399c33e7 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:48.142444 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:48.154941 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcceed1-e3da-444c-b349-b2264686bdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:48.155988 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcceed1-e3da-444c-b349-b2264686bdfc None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:48.156503 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcceed1-e3da-444c-b349-b2264686bdfc None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:48.157104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcceed1-e3da-444c-b349-b2264686bdfc None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:48.158186 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcceed1-e3da-444c-b349-b2264686bdfc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:48.158703 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcceed1-e3da-444c-b349-b2264686bdfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:48.160392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcceed1-e3da-444c-b349-b2264686bdfc None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:48.171962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 89/173] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:48:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:48.173612 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad349b-f53f-4f04-ae7c-f9126e7eb4d7 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:48.174123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad349b-f53f-4f04-ae7c-f9126e7eb4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:48.174593 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad349b-f53f-4f04-ae7c-f9126e7eb4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:48.195989 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a9ad349b-f53f-4f04-ae7c-f9126e7eb4d7 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:48.198390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a9ad349b-f53f-4f04-ae7c-f9126e7eb4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:48.288028 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 90/174] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:48.289344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad349b-f53f-4f04-ae7c-f9126e7eb4d7 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:48.339604 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15936fcf-75a2-48cf-9b6a-c9f2399c33e7 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:48.418508 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f5545-c3b5-4a3c-b060-404e57939636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:48.419823 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f5545-c3b5-4a3c-b060-404e57939636 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:48.420307 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f5545-c3b5-4a3c-b060-404e57939636 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:48.420800 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f5545-c3b5-4a3c-b060-404e57939636 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:48.423124 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f5545-c3b5-4a3c-b060-404e57939636 None admin] RBAC: Adding query filter params (role.id=0efeff46573247f3b9f1e2dec2968672, scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532, user.id=2e3ec4696fd447c89f8d404bb4540c6c) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:48.423689 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f5545-c3b5-4a3c-b060-404e57939636 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:48.427687 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f5545-c3b5-4a3c-b060-404e57939636 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:48.444759 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 85/175] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:48:48 2022] GET /identity/v3/role_assignments?user.id=2e3ec4696fd447c89f8d404bb4540c6c&scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532&role.id=0efeff46573247f3b9f1e2dec2968672 => generated 608 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:48.446283 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f5545-c3b5-4a3c-b060-404e57939636 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:48.447609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad349b-f53f-4f04-ae7c-f9126e7eb4d7 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:48.491267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:48.542015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387518f-3279-41dc-a6f3-edafffc4892a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:48.543380 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387518f-3279-41dc-a6f3-edafffc4892a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:48.543900 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387518f-3279-41dc-a6f3-edafffc4892a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:48.545263 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387518f-3279-41dc-a6f3-edafffc4892a None admin] PATH_INFO: `/v3/users/bcbfd29852d047909a1de71fb64876b1` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:48.547366 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6387518f-3279-41dc-a6f3-edafffc4892a None admin] RBAC: Authorizing `identity:get_user(user_id=bcbfd29852d047909a1de71fb64876b1)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:48.550589 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6387518f-3279-41dc-a6f3-edafffc4892a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:48.557369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 91/176] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:48 2022] GET /identity/v3/users/bcbfd29852d047909a1de71fb64876b1 => generated 275 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:48.560038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387518f-3279-41dc-a6f3-edafffc4892a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:48.573763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:48.719064 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f5545-c3b5-4a3c-b060-404e57939636 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:48.753970 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387518f-3279-41dc-a6f3-edafffc4892a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:48.783172 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ac4fc-242e-474a-88e2-d6589b7fff0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:48.785321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ac4fc-242e-474a-88e2-d6589b7fff0c None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:48.785838 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ac4fc-242e-474a-88e2-d6589b7fff0c None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:48.786336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ac4fc-242e-474a-88e2-d6589b7fff0c None admin] PATH_INFO: `/v3/users/2f686cad97234a329e2f1f46b2f6258b` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:48.789497 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ac4fc-242e-474a-88e2-d6589b7fff0c None admin] RBAC: Authorizing `identity:get_user(user_id=2f686cad97234a329e2f1f46b2f6258b)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:48.793823 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ac4fc-242e-474a-88e2-d6589b7fff0c None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:48.797895 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 86/177] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:48 2022] GET /identity/v3/users/2f686cad97234a329e2f1f46b2f6258b => generated 289 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:48.798486 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ac4fc-242e-474a-88e2-d6589b7fff0c None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:48.803637 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1de57-6d90-4584-8ed1-9588b7814657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:48.804915 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1de57-6d90-4584-8ed1-9588b7814657 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:48.805341 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1de57-6d90-4584-8ed1-9588b7814657 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:48.805800 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1de57-6d90-4584-8ed1-9588b7814657 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:48.825999 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1de57-6d90-4584-8ed1-9588b7814657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:48.843642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-02d1de57-6d90-4584-8ed1-9588b7814657 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:48:48.843642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:48.843642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:48.843642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:48.843642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:48.843642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:48.843642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:48.843642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:48.843642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:48.843642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:48.843642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:48.843642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:48.843642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:48.843642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:48.843642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:48.843642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:48.844715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:48.844715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:48.844715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:48.844715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:48.844715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:48.844715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:48.844715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:48.844715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:48.844715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:48.844715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:48.844715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:48.844715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:48.844715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:48.844715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:48.844715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:48.844715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:48.845613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:48.845613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:48.845613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:48.845613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:48.845613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:48.845613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:48.845613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:48.845613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:48.845613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:48.845613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:48.845613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:48:48.845613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:48:48.846284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 92/178] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:48:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:48.851991 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033eabb-117f-4cd4-b609-17660fa03244 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:48.852471 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033eabb-117f-4cd4-b609-17660fa03244 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:48.853610 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033eabb-117f-4cd4-b609-17660fa03244 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:48.856509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 93/179] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:48 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:48.857900 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2033eabb-117f-4cd4-b609-17660fa03244 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:48.959792 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:48.965697 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ac4fc-242e-474a-88e2-d6589b7fff0c None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:48.984336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:49.019702 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36ee67-3766-4372-9835-5f09c2f9df1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:49.020830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36ee67-3766-4372-9835-5f09c2f9df1a None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:49.021787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36ee67-3766-4372-9835-5f09c2f9df1a None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:49.022500 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36ee67-3766-4372-9835-5f09c2f9df1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:49.068280 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2033eabb-117f-4cd4-b609-17660fa03244 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:49.115492 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36ee67-3766-4372-9835-5f09c2f9df1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:49.117058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36ee67-3766-4372-9835-5f09c2f9df1a None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:49.131757 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08fbdd-dd9c-427e-8d3a-c4be026afcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:49.133303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08fbdd-dd9c-427e-8d3a-c4be026afcbb None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:49.133831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08fbdd-dd9c-427e-8d3a-c4be026afcbb None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:49.135076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08fbdd-dd9c-427e-8d3a-c4be026afcbb None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:49.137253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08fbdd-dd9c-427e-8d3a-c4be026afcbb None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:49.140291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08fbdd-dd9c-427e-8d3a-c4be026afcbb None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:49.144300 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 94/180] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:48:48 2022] GET /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532 => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:49.145813 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08fbdd-dd9c-427e-8d3a-c4be026afcbb None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:49.265676 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 87/181] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:48:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:48:49.265676 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0c671-2f40-471f-84c4-299ac53ae9d7 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:49.265676 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0c671-2f40-471f-84c4-299ac53ae9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:49.265676 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0c671-2f40-471f-84c4-299ac53ae9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:49.284434 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-dae0c671-2f40-471f-84c4-299ac53ae9d7 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:49.285690 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-dae0c671-2f40-471f-84c4-299ac53ae9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:49.293447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08fbdd-dd9c-427e-8d3a-c4be026afcbb None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:49.295368 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:49.307372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:49.334818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a483457-5d0a-4301-b6c2-e6acd929047b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:49.335900 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a483457-5d0a-4301-b6c2-e6acd929047b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:49.336353 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a483457-5d0a-4301-b6c2-e6acd929047b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:49.336788 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a483457-5d0a-4301-b6c2-e6acd929047b None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:49.338410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a483457-5d0a-4301-b6c2-e6acd929047b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:49.338971 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a483457-5d0a-4301-b6c2-e6acd929047b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:49.342085 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a483457-5d0a-4301-b6c2-e6acd929047b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:49.354936 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 95/182] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:48:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:49.362586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3290b7-a8c8-4c16-9bd7-a4c40e6857f8 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:49.362586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3290b7-a8c8-4c16-9bd7-a4c40e6857f8 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:49.362586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3290b7-a8c8-4c16-9bd7-a4c40e6857f8 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:49.362586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 96/183] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:49.363696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3290b7-a8c8-4c16-9bd7-a4c40e6857f8 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:49.378816 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 88/184] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:49.380047 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0c671-2f40-471f-84c4-299ac53ae9d7 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:49.518218 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3290b7-a8c8-4c16-9bd7-a4c40e6857f8 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:49.523754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0c671-2f40-471f-84c4-299ac53ae9d7 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:49.563879 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569ca22-26f1-43bb-a69a-03dfde5b2475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:49.564766 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569ca22-26f1-43bb-a69a-03dfde5b2475 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:49.565043 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569ca22-26f1-43bb-a69a-03dfde5b2475 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:49.565317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569ca22-26f1-43bb-a69a-03dfde5b2475 None admin] PATH_INFO: `/v3/projects/7f20b07ad9f648969064ba9000eb5a8e` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:49.566540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569ca22-26f1-43bb-a69a-03dfde5b2475 None admin] RBAC: Authorizing `identity:get_project(project_id=7f20b07ad9f648969064ba9000eb5a8e)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:49.568057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569ca22-26f1-43bb-a69a-03dfde5b2475 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:49.569786 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 89/185] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:48:49 2022] GET /identity/v3/projects/7f20b07ad9f648969064ba9000eb5a8e => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:49.570559 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569ca22-26f1-43bb-a69a-03dfde5b2475 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:49.576117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a8682-50f7-4d15-b0e2-0b485814b7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:49.578006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a8682-50f7-4d15-b0e2-0b485814b7d7 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:49.578503 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a8682-50f7-4d15-b0e2-0b485814b7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:49.579041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a8682-50f7-4d15-b0e2-0b485814b7d7 None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532/users/2f686cad97234a329e2f1f46b2f6258b/roles/7f966f669d82448daddfe911e2955ea2` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:49.582564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a8682-50f7-4d15-b0e2-0b485814b7d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532, user_id=2f686cad97234a329e2f1f46b2f6258b, role_id=7f966f669d82448daddfe911e2955ea2)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:49.584803 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a8682-50f7-4d15-b0e2-0b485814b7d7 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:49.589789 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:49.682138 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569ca22-26f1-43bb-a69a-03dfde5b2475 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:49.710486 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 97/186] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:48:49 2022] PUT /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532/users/2f686cad97234a329e2f1f46b2f6258b/roles/7f966f669d82448daddfe911e2955ea2 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:48:49.713485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c097a7-b528-40fe-b757-7ccd434039bb None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:49.714079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c097a7-b528-40fe-b757-7ccd434039bb None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:49.714605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c097a7-b528-40fe-b757-7ccd434039bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:49.725279 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d459a-f82d-4f59-8283-ea80c3cc9591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:49.726319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d459a-f82d-4f59-8283-ea80c3cc9591 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:49.726686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d459a-f82d-4f59-8283-ea80c3cc9591 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:49.727122 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d459a-f82d-4f59-8283-ea80c3cc9591 None admin] PATH_INFO: `/v3/users/4b51fbf7a8cf4b3ab54456f31a074a6f` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:49.746096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d459a-f82d-4f59-8283-ea80c3cc9591 None admin] RBAC: Authorizing `identity:get_user(user_id=4b51fbf7a8cf4b3ab54456f31a074a6f)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:49.755648 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d459a-f82d-4f59-8283-ea80c3cc9591 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:49.755648 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 90/187] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:49 2022] GET /identity/v3/users/4b51fbf7a8cf4b3ab54456f31a074a6f => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:49.755648 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d459a-f82d-4f59-8283-ea80c3cc9591 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:49.758079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-58c097a7-b528-40fe-b757-7ccd434039bb None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:49.760055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-58c097a7-b528-40fe-b757-7ccd434039bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:49.813622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:49.861247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:49.894183 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 98/188] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:49.895947 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c097a7-b528-40fe-b757-7ccd434039bb None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:49.932524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d459a-f82d-4f59-8283-ea80c3cc9591 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:49.986015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9d34c-9c30-48b6-9742-9cc1b73a172d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:49.987371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9d34c-9c30-48b6-9742-9cc1b73a172d None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:49.987371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9d34c-9c30-48b6-9742-9cc1b73a172d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:49.987660 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9d34c-9c30-48b6-9742-9cc1b73a172d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:50.000515 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e9d34c-9c30-48b6-9742-9cc1b73a172d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:50.008810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-c9e9d34c-9c30-48b6-9742-9cc1b73a172d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:48:50.008810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:50.008810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:50.008810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:50.008810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:50.008810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:50.008810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:50.008810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:50.008810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:50.008810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:50.008810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:50.008810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:50.008810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:50.008810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:50.008810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:50.008810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:50.011120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:50.011120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:50.011120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:50.011120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:50.011120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:50.011120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:50.011120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:50.011120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:50.011120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:50.011120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:50.011120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:50.011120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:50.011120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:50.011120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:50.011120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:50.011120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:50.012558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:50.012558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:50.012558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:50.012558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:50.012558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:50.012558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:50.012558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:50.012558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:50.012558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:50.012558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:50.012558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:48:50.012558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:48:50.012558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 91/189] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:48:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:50.012558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9d34c-9c30-48b6-9742-9cc1b73a172d None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:50.096784 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c097a7-b528-40fe-b757-7ccd434039bb None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:50.147264 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2d6b6-3733-49f9-a59c-cec4c29101da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:50.148305 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2d6b6-3733-49f9-a59c-cec4c29101da None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:50.148822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2d6b6-3733-49f9-a59c-cec4c29101da None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:50.149273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2d6b6-3733-49f9-a59c-cec4c29101da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:50.151358 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2d6b6-3733-49f9-a59c-cec4c29101da None admin] RBAC: Adding query filter params (role.id=b7f92100b08f4dc6b2b6866a2955d2ab, scope.project.id=7f20b07ad9f648969064ba9000eb5a8e, user.id=bcbfd29852d047909a1de71fb64876b1) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:50.151908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2d6b6-3733-49f9-a59c-cec4c29101da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:50.154275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2d6b6-3733-49f9-a59c-cec4c29101da None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:50.165726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 99/190] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:48:49 2022] GET /identity/v3/role_assignments?user.id=bcbfd29852d047909a1de71fb64876b1&scope.project.id=7f20b07ad9f648969064ba9000eb5a8e&role.id=b7f92100b08f4dc6b2b6866a2955d2ab => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:50.167634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2d6b6-3733-49f9-a59c-cec4c29101da None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:50.202361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9d34c-9c30-48b6-9742-9cc1b73a172d None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:50.226662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:50.263488 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:50.283012 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa1136-4425-4445-a0c4-682ea3783869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:50.285320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa1136-4425-4445-a0c4-682ea3783869 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:50.286123 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa1136-4425-4445-a0c4-682ea3783869 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:50.286790 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa1136-4425-4445-a0c4-682ea3783869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:50.380708 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2d6b6-3733-49f9-a59c-cec4c29101da None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:50.449017 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa1136-4425-4445-a0c4-682ea3783869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:50.452940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa1136-4425-4445-a0c4-682ea3783869 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:50.470674 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34924d-fa1b-4796-b797-cf29dbf4fccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:50.471425 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34924d-fa1b-4796-b797-cf29dbf4fccb None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:50.471906 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34924d-fa1b-4796-b797-cf29dbf4fccb None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:50.472375 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34924d-fa1b-4796-b797-cf29dbf4fccb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:50.475501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34924d-fa1b-4796-b797-cf29dbf4fccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:50.478780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34924d-fa1b-4796-b797-cf29dbf4fccb None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:50.483106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 100/191] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:48:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:50.484981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34924d-fa1b-4796-b797-cf29dbf4fccb None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:50.650106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:50.657651 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 92/192] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:48:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:48:50.657651 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa1136-4425-4445-a0c4-682ea3783869 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:50.667301 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34924d-fa1b-4796-b797-cf29dbf4fccb None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:50.706579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:50.725464 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03af28e-fe35-405d-ae0d-918b1af981be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:50.726364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03af28e-fe35-405d-ae0d-918b1af981be None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:50.726765 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03af28e-fe35-405d-ae0d-918b1af981be None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:50.727206 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03af28e-fe35-405d-ae0d-918b1af981be None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:50.730715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03af28e-fe35-405d-ae0d-918b1af981be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:50.730715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03af28e-fe35-405d-ae0d-918b1af981be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:50.730715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03af28e-fe35-405d-ae0d-918b1af981be None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:50.742999 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 101/193] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:48:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:50.748506 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03af28e-fe35-405d-ae0d-918b1af981be None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:50.836474 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa1136-4425-4445-a0c4-682ea3783869 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:50.877215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fe42a-4f64-4de8-8405-d073c2c1bf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:50.883500 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fe42a-4f64-4de8-8405-d073c2c1bf63 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:50.883500 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fe42a-4f64-4de8-8405-d073c2c1bf63 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:50.883500 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fe42a-4f64-4de8-8405-d073c2c1bf63 None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:50.883500 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169fe42a-4f64-4de8-8405-d073c2c1bf63 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:50.886215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169fe42a-4f64-4de8-8405-d073c2c1bf63 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:50.937819 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03af28e-fe35-405d-ae0d-918b1af981be None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:50.999605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41452f3-5559-4c9a-9f6c-da6c8481d0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:51.001163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41452f3-5559-4c9a-9f6c-da6c8481d0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:51.001665 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41452f3-5559-4c9a-9f6c-da6c8481d0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:51.002069 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41452f3-5559-4c9a-9f6c-da6c8481d0b1 None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:51.004287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41452f3-5559-4c9a-9f6c-da6c8481d0b1 None admin] RBAC: Authorizing `identity:get_project(project_id=aad3af41314e48b4a48d335366f333ce)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:51.007174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41452f3-5559-4c9a-9f6c-da6c8481d0b1 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:51.017148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 93/194] 10.222.0.27 () {60 vars in 1132 bytes} [Sat Jul 16 14:48:50 2022] POST /identity/v3/users => generated 297 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:48:51.017364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 102/195] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:48:50 2022] GET /identity/v3/projects/aad3af41314e48b4a48d335366f333ce => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:51.018379 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fe42a-4f64-4de8-8405-d073c2c1bf63 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:51.020111 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41452f3-5559-4c9a-9f6c-da6c8481d0b1 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:51.069320 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:51.201750 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:51.205313 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41452f3-5559-4c9a-9f6c-da6c8481d0b1 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:51.237678 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fe42a-4f64-4de8-8405-d073c2c1bf63 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:51.278762 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68081488-a05b-4f8e-ae96-0579dd1fb53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:51.280084 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68081488-a05b-4f8e-ae96-0579dd1fb53c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:51.280561 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68081488-a05b-4f8e-ae96-0579dd1fb53c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:51.281534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68081488-a05b-4f8e-ae96-0579dd1fb53c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:51.283529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68081488-a05b-4f8e-ae96-0579dd1fb53c None admin] RBAC: Adding query filter params (role.id=0efeff46573247f3b9f1e2dec2968672, scope.project.id=aad3af41314e48b4a48d335366f333ce, user.id=4b51fbf7a8cf4b3ab54456f31a074a6f) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:51.284036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68081488-a05b-4f8e-ae96-0579dd1fb53c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:51.287716 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68081488-a05b-4f8e-ae96-0579dd1fb53c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:51.296466 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f9c62-6af3-4490-84a2-068a8ba73b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:51.297755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f9c62-6af3-4490-84a2-068a8ba73b8f None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:51.298175 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f9c62-6af3-4490-84a2-068a8ba73b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:51.298618 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f9c62-6af3-4490-84a2-068a8ba73b8f None admin] PATH_INFO: `/v3/users/4b51fbf7a8cf4b3ab54456f31a074a6f` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:51.299315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 103/196] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:48:51 2022] GET /identity/v3/role_assignments?user.id=4b51fbf7a8cf4b3ab54456f31a074a6f&scope.project.id=aad3af41314e48b4a48d335366f333ce&role.id=0efeff46573247f3b9f1e2dec2968672 => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:51.300314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f9c62-6af3-4490-84a2-068a8ba73b8f None admin] RBAC: Authorizing `identity:get_user(user_id=4b51fbf7a8cf4b3ab54456f31a074a6f)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:51.301967 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f9c62-6af3-4490-84a2-068a8ba73b8f None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:51.303912 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 94/197] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:51 2022] GET /identity/v3/users/4b51fbf7a8cf4b3ab54456f31a074a6f => generated 283 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:51.312168 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68081488-a05b-4f8e-ae96-0579dd1fb53c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:51.453039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:51.473001 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68081488-a05b-4f8e-ae96-0579dd1fb53c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:51.526945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f888f2-3daf-45cd-b38b-8dfa2d44b820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:51.527629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f888f2-3daf-45cd-b38b-8dfa2d44b820 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:51.527914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f888f2-3daf-45cd-b38b-8dfa2d44b820 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:51.528262 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f888f2-3daf-45cd-b38b-8dfa2d44b820 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:51.607042 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f888f2-3daf-45cd-b38b-8dfa2d44b820 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:51.608563 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f888f2-3daf-45cd-b38b-8dfa2d44b820 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:51.715772 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:51.728082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 104/198] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:48:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:48:51.733993 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f9c62-6af3-4490-84a2-068a8ba73b8f None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:51.927237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f9c62-6af3-4490-84a2-068a8ba73b8f None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:52.001824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb782ee1-680a-45c3-8d32-9313563370b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:52.003227 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb782ee1-680a-45c3-8d32-9313563370b6 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:52.003706 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb782ee1-680a-45c3-8d32-9313563370b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:52.004176 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb782ee1-680a-45c3-8d32-9313563370b6 None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:52.012470 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb782ee1-680a-45c3-8d32-9313563370b6 None admin] RBAC: Authorizing `identity:get_project(project_id=aad3af41314e48b4a48d335366f333ce)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:52.015205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb782ee1-680a-45c3-8d32-9313563370b6 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:52.018954 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 95/199] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:48:51 2022] GET /identity/v3/projects/aad3af41314e48b4a48d335366f333ce => generated 302 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:48:52.025254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f888f2-3daf-45cd-b38b-8dfa2d44b820 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:52.180589 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f888f2-3daf-45cd-b38b-8dfa2d44b820 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:52.238262 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764c1e7-2b90-400a-9616-3969cc797fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:52.239261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764c1e7-2b90-400a-9616-3969cc797fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:52.239261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764c1e7-2b90-400a-9616-3969cc797fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:52.239491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764c1e7-2b90-400a-9616-3969cc797fb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:52.240374 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764c1e7-2b90-400a-9616-3969cc797fb9 None admin] RBAC: Adding query filter params (role.id=af11cd5265d14ac7902b1fb3afcc4be7, scope.project.id=aad3af41314e48b4a48d335366f333ce, user.id=4b51fbf7a8cf4b3ab54456f31a074a6f) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:52.240688 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764c1e7-2b90-400a-9616-3969cc797fb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:52.242226 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764c1e7-2b90-400a-9616-3969cc797fb9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:52.249873 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 105/200] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:48:52 2022] GET /identity/v3/role_assignments?user.id=4b51fbf7a8cf4b3ab54456f31a074a6f&scope.project.id=aad3af41314e48b4a48d335366f333ce&role.id=af11cd5265d14ac7902b1fb3afcc4be7 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:48:52.625079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bbe36-5b70-4031-b8d3-c0537f2f6f65 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:52.625544 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bbe36-5b70-4031-b8d3-c0537f2f6f65 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:52.625906 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bbe36-5b70-4031-b8d3-c0537f2f6f65 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:52.629504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 96/201] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:52.637906 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97b9df-6704-453a-a40a-8b5a3e482dc7 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:52.638961 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97b9df-6704-453a-a40a-8b5a3e482dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:52.639381 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97b9df-6704-453a-a40a-8b5a3e482dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:52.678045 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-1d97b9df-6704-453a-a40a-8b5a3e482dc7 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:52.679882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1d97b9df-6704-453a-a40a-8b5a3e482dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:52.783442 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:52.821282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 106/202] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:52.872572 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bbe36-5b70-4031-b8d3-c0537f2f6f65 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:52.945606 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:53.126129 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bbe36-5b70-4031-b8d3-c0537f2f6f65 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:53.186506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc647a4-265b-4267-84d1-16fd99c904c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:53.186506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc647a4-265b-4267-84d1-16fd99c904c1 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:53.186860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc647a4-265b-4267-84d1-16fd99c904c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:53.187017 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc647a4-265b-4267-84d1-16fd99c904c1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:53.205531 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc647a4-265b-4267-84d1-16fd99c904c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:53.215540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-0dc647a4-265b-4267-84d1-16fd99c904c1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 14:48:53.215540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:53.215540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:53.215540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:53.215540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:53.215540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:53.215540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:53.215540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:53.215540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:53.215540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:53.215540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:53.215540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:53.215540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:53.215540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:53.215540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:53.215540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:53.217213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:53.217213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:53.217213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:53.217213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:53.217213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:53.217213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:53.217213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:53.217213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:53.217213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:53.217213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:53.217213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:53.217213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:53.217213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:53.217213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:53.217213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:53.217213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:53.218821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:53.218821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:53.218821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:53.218821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:53.218821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:53.218821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:53.218821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:53.218821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:53.218821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:53.218821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:53.218821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 14:48:53.218821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:48:53.218821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 97/203] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:48:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:53.228731 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97b9df-6704-453a-a40a-8b5a3e482dc7 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:53.452847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97b9df-6704-453a-a40a-8b5a3e482dc7 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:53.512736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df13f0-3c3c-40c4-9f31-abdc48fc4b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:53.513437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df13f0-3c3c-40c4-9f31-abdc48fc4b83 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:53.513717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df13f0-3c3c-40c4-9f31-abdc48fc4b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:53.513986 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df13f0-3c3c-40c4-9f31-abdc48fc4b83 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:53.514760 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df13f0-3c3c-40c4-9f31-abdc48fc4b83 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:53.515084 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df13f0-3c3c-40c4-9f31-abdc48fc4b83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:53.516503 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df13f0-3c3c-40c4-9f31-abdc48fc4b83 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:53.531942 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 107/204] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:48:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:53.537809 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc647a4-265b-4267-84d1-16fd99c904c1 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:53.709808 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:53.734634 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc647a4-265b-4267-84d1-16fd99c904c1 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:53.784067 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe36ca-b2df-4c16-8cee-282674532c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:53.785457 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe36ca-b2df-4c16-8cee-282674532c57 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:53.785962 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe36ca-b2df-4c16-8cee-282674532c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:53.786435 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe36ca-b2df-4c16-8cee-282674532c57 None admin] PATH_INFO: `/v3/users/2f686cad97234a329e2f1f46b2f6258b` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:53.788680 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe36ca-b2df-4c16-8cee-282674532c57 None admin] RBAC: Authorizing `identity:get_user(user_id=2f686cad97234a329e2f1f46b2f6258b)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:53.791192 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe36ca-b2df-4c16-8cee-282674532c57 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:53.793146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 98/205] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:53 2022] GET /identity/v3/users/2f686cad97234a329e2f1f46b2f6258b => generated 289 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:53.802151 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df13f0-3c3c-40c4-9f31-abdc48fc4b83 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:53.906419 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:54.051068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df13f0-3c3c-40c4-9f31-abdc48fc4b83 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:54.124412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28066a20-4161-433c-8bcc-59ae61b5d680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:54.125104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28066a20-4161-433c-8bcc-59ae61b5d680 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:54.125390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28066a20-4161-433c-8bcc-59ae61b5d680 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:54.125675 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28066a20-4161-433c-8bcc-59ae61b5d680 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:54.220029 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa370d5-9de3-41af-a895-e3f0889de6ef None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:54.221214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa370d5-9de3-41af-a895-e3f0889de6ef None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:54.223124 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa370d5-9de3-41af-a895-e3f0889de6ef None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:54.224817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 99/206] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:54.232246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521f027-3cbb-42b6-91da-0ab03f925318 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:54.233078 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521f027-3cbb-42b6-91da-0ab03f925318 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:54.234945 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521f027-3cbb-42b6-91da-0ab03f925318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:54.262355 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28066a20-4161-433c-8bcc-59ae61b5d680 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:54.263905 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28066a20-4161-433c-8bcc-59ae61b5d680 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:54.271939 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-0521f027-3cbb-42b6-91da-0ab03f925318 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:54.273193 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-0521f027-3cbb-42b6-91da-0ab03f925318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:54.336595 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:54.376923 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 100/207] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:54.379794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce8c20-9d2a-46b1-b269-047f1f8b1edd None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:54.380336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce8c20-9d2a-46b1-b269-047f1f8b1edd None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:54.380830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce8c20-9d2a-46b1-b269-047f1f8b1edd None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:54.382738 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 101/208] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:54.401485 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71653a11-b0c9-4dda-afa4-7de0e709b8bc None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:54.402142 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71653a11-b0c9-4dda-afa4-7de0e709b8bc None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:54.402613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71653a11-b0c9-4dda-afa4-7de0e709b8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:54.416897 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 108/209] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:48:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:48:54.426169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28066a20-4161-433c-8bcc-59ae61b5d680 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:54.435643 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-71653a11-b0c9-4dda-afa4-7de0e709b8bc None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:54.436857 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-71653a11-b0c9-4dda-afa4-7de0e709b8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:54.572583 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 102/210] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:54.575579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71653a11-b0c9-4dda-afa4-7de0e709b8bc None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:54.592438 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28066a20-4161-433c-8bcc-59ae61b5d680 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:54.597104 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:54.636272 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226df82b-7102-4c25-93fd-ba635e739f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:54.637402 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226df82b-7102-4c25-93fd-ba635e739f2d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:54.637895 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226df82b-7102-4c25-93fd-ba635e739f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:54.638382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226df82b-7102-4c25-93fd-ba635e739f2d None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:54.639924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226df82b-7102-4c25-93fd-ba635e739f2d None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:54.641603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226df82b-7102-4c25-93fd-ba635e739f2d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:54.644073 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 109/211] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:48:54 2022] GET /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:54.650500 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226df82b-7102-4c25-93fd-ba635e739f2d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:54.675319 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:54.706624 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71653a11-b0c9-4dda-afa4-7de0e709b8bc None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:54.751657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd89ed-2623-418c-8a8b-73d096d58d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:54.752539 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd89ed-2623-418c-8a8b-73d096d58d05 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:54.752853 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd89ed-2623-418c-8a8b-73d096d58d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:54.753138 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd89ed-2623-418c-8a8b-73d096d58d05 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:54.769858 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbd89ed-2623-418c-8a8b-73d096d58d05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:54.785875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-0dbd89ed-2623-418c-8a8b-73d096d58d05 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:48:54.785875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:54.785875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:54.785875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:54.785875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:54.785875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:54.785875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:54.785875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:54.785875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:54.785875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:54.785875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:54.785875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:54.785875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:54.785875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:54.785875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:54.785875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:54.786848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:54.786848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:54.786848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:54.786848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:54.786848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:54.786848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:54.786848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:54.786848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:54.786848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:54.786848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:54.786848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:54.786848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:54.786848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:54.786848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:54.786848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:54.786848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:54.787675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:54.787675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:54.787675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:54.787675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:54.787675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:54.787675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:54.787675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:54.787675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:54.787675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:54.787675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:54.787675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:48:54.787675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:48:54.788839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 103/212] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:48:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:54.790175 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd89ed-2623-418c-8a8b-73d096d58d05 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:54.808901 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226df82b-7102-4c25-93fd-ba635e739f2d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:54.864493 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e2b4b-b1be-4764-84f1-df1df5234462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:54.865802 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e2b4b-b1be-4764-84f1-df1df5234462 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:54.866249 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e2b4b-b1be-4764-84f1-df1df5234462 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:54.866709 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e2b4b-b1be-4764-84f1-df1df5234462 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:54.885081 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:54.887000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0e2b4b-b1be-4764-84f1-df1df5234462 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:54.902649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-7f0e2b4b-b1be-4764-84f1-df1df5234462 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:48:54.902649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:54.902649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:54.902649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:54.902649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:54.902649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:54.902649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:54.902649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:54.902649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:54.902649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:54.902649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:54.902649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:54.902649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:54.902649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:54.902649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:54.902649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:54.911344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:54.911344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:54.911344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:54.911344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:54.911344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:54.911344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:54.911344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:54.911344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:54.911344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:54.911344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:54.911344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:54.911344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:54.911344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:54.911344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:54.911344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:54.911344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:54.912145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:54.912145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:54.912145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:54.912145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:54.912145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:54.912145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:54.912145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:54.912145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:54.912145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:54.912145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:54.912145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:48:54.912145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:48:54.912145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 110/213] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:48:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:54.912145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae46c54-12d1-4492-9bfb-b7dbe6ed522e None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:54.912846 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae46c54-12d1-4492-9bfb-b7dbe6ed522e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:54.912846 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae46c54-12d1-4492-9bfb-b7dbe6ed522e None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:54.912846 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 111/214] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:54.912846 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae46c54-12d1-4492-9bfb-b7dbe6ed522e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:54.950724 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd89ed-2623-418c-8a8b-73d096d58d05 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:55.006039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:55.017071 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af54ccf-fdad-436b-b704-64e81e5d47a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:55.017754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af54ccf-fdad-436b-b704-64e81e5d47a4 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:55.018182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af54ccf-fdad-436b-b704-64e81e5d47a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:55.021774 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af54ccf-fdad-436b-b704-64e81e5d47a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:55.021774 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af54ccf-fdad-436b-b704-64e81e5d47a4 None admin] RBAC: Adding query filter params (role.id=7f966f669d82448daddfe911e2955ea2, scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532, user.id=2f686cad97234a329e2f1f46b2f6258b) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:55.021774 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af54ccf-fdad-436b-b704-64e81e5d47a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:55.021774 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af54ccf-fdad-436b-b704-64e81e5d47a4 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:55.038946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 104/215] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:48:54 2022] GET /identity/v3/role_assignments?user.id=2f686cad97234a329e2f1f46b2f6258b&scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532&role.id=7f966f669d82448daddfe911e2955ea2 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:55.038946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af54ccf-fdad-436b-b704-64e81e5d47a4 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:55.060138 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae46c54-12d1-4492-9bfb-b7dbe6ed522e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:55.103285 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03080c0c-3e6f-4c7d-8d5d-07f4306a5804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:55.106201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03080c0c-3e6f-4c7d-8d5d-07f4306a5804 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:55.106201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03080c0c-3e6f-4c7d-8d5d-07f4306a5804 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:55.106201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03080c0c-3e6f-4c7d-8d5d-07f4306a5804 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:55.106201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03080c0c-3e6f-4c7d-8d5d-07f4306a5804 None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:55.106201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03080c0c-3e6f-4c7d-8d5d-07f4306a5804 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:55.107589 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03080c0c-3e6f-4c7d-8d5d-07f4306a5804 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:55.118029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 112/216] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:48:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:55.119993 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5f66e-94e8-40ae-8627-2f9ec344725d None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:55.120283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5f66e-94e8-40ae-8627-2f9ec344725d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:55.120563 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5f66e-94e8-40ae-8627-2f9ec344725d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:55.142913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a0a5f66e-94e8-40ae-8627-2f9ec344725d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:55.144311 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a0a5f66e-94e8-40ae-8627-2f9ec344725d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:55.231682 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af54ccf-fdad-436b-b704-64e81e5d47a4 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:55.245205 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 113/217] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:55.246803 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5f66e-94e8-40ae-8627-2f9ec344725d None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:55.301388 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:55.316147 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24773a52-a0ae-4cf7-85e6-ce9458d4a56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:55.317292 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24773a52-a0ae-4cf7-85e6-ce9458d4a56c None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:55.317803 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24773a52-a0ae-4cf7-85e6-ce9458d4a56c None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:55.318319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24773a52-a0ae-4cf7-85e6-ce9458d4a56c None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:55.319852 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24773a52-a0ae-4cf7-85e6-ce9458d4a56c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:55.320418 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24773a52-a0ae-4cf7-85e6-ce9458d4a56c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:55.323830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24773a52-a0ae-4cf7-85e6-ce9458d4a56c None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:55.344889 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 105/218] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:48:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:55.344889 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24773a52-a0ae-4cf7-85e6-ce9458d4a56c None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:55.371308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:55.441447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5f66e-94e8-40ae-8627-2f9ec344725d None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:55.484335 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24773a52-a0ae-4cf7-85e6-ce9458d4a56c None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:55.489829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec58ba-2753-4434-9b50-19b7ac6c3e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:55.490721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec58ba-2753-4434-9b50-19b7ac6c3e8f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:55.491055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec58ba-2753-4434-9b50-19b7ac6c3e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:55.491296 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec58ba-2753-4434-9b50-19b7ac6c3e8f None admin] PATH_INFO: `/v3/users/d0b6e49f4c414f668e6e44be04eba19d` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:55.509844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec58ba-2753-4434-9b50-19b7ac6c3e8f None admin] RBAC: Authorizing `identity:get_user(user_id=d0b6e49f4c414f668e6e44be04eba19d)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:55.511719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec58ba-2753-4434-9b50-19b7ac6c3e8f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:55.513799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 114/219] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:55 2022] GET /identity/v3/users/d0b6e49f4c414f668e6e44be04eba19d => generated 297 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:55.514360 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec58ba-2753-4434-9b50-19b7ac6c3e8f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:55.533444 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6375ae-0bfb-436f-882b-da55894950f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:55.534290 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6375ae-0bfb-436f-882b-da55894950f5 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:55.534567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6375ae-0bfb-436f-882b-da55894950f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:55.534837 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6375ae-0bfb-436f-882b-da55894950f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:55.536135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6375ae-0bfb-436f-882b-da55894950f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:55.537623 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6375ae-0bfb-436f-882b-da55894950f5 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:55.539379 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 106/220] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:48:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:55.540212 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6375ae-0bfb-436f-882b-da55894950f5 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:55.613527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:55.661458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec58ba-2753-4434-9b50-19b7ac6c3e8f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:55.675436 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6375ae-0bfb-436f-882b-da55894950f5 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:55.705886 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:55.725144 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7aaad-9908-4b39-b7e7-2ebf57aaceb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:55.726579 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7aaad-9908-4b39-b7e7-2ebf57aaceb5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:55.727090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7aaad-9908-4b39-b7e7-2ebf57aaceb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:55.728261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7aaad-9908-4b39-b7e7-2ebf57aaceb5 None admin] PATH_INFO: `/v3/users/4b51fbf7a8cf4b3ab54456f31a074a6f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:55.730278 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7aaad-9908-4b39-b7e7-2ebf57aaceb5 None admin] RBAC: Authorizing `identity:get_user(user_id=4b51fbf7a8cf4b3ab54456f31a074a6f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:55.730841 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c1be9-b6c5-473f-81ae-c313e1d846dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:55.731928 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c1be9-b6c5-473f-81ae-c313e1d846dd None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:55.732383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c1be9-b6c5-473f-81ae-c313e1d846dd None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:55.732823 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c1be9-b6c5-473f-81ae-c313e1d846dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:55.732913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7aaad-9908-4b39-b7e7-2ebf57aaceb5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:55.738059 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 115/221] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:55 2022] GET /identity/v3/users/4b51fbf7a8cf4b3ab54456f31a074a6f => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:55.740454 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7aaad-9908-4b39-b7e7-2ebf57aaceb5 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:55.847627 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c1be9-b6c5-473f-81ae-c313e1d846dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:55.849385 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c1be9-b6c5-473f-81ae-c313e1d846dd None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:55.893853 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7aaad-9908-4b39-b7e7-2ebf57aaceb5 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:55.933162 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:55.940953 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a552096-2fc0-4808-9823-2ae53f0601e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:55.941793 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a552096-2fc0-4808-9823-2ae53f0601e8 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:55.942242 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a552096-2fc0-4808-9823-2ae53f0601e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:55.946974 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a552096-2fc0-4808-9823-2ae53f0601e8 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:55.946974 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a552096-2fc0-4808-9823-2ae53f0601e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:55.946974 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a552096-2fc0-4808-9823-2ae53f0601e8 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:55.989790 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 107/222] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:48:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:48:55.990997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c1be9-b6c5-473f-81ae-c313e1d846dd None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:56.037341 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 116/223] 10.222.0.27 () {60 vars in 1132 bytes} [Sat Jul 16 14:48:55 2022] POST /identity/v3/users => generated 297 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:48:56.037341 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b6fed-92a4-4c2d-b33a-f9c9e32b79f8 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:56.037341 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b6fed-92a4-4c2d-b33a-f9c9e32b79f8 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:56.037341 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b6fed-92a4-4c2d-b33a-f9c9e32b79f8 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:56.037341 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 117/224] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:48:56.037341 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b6fed-92a4-4c2d-b33a-f9c9e32b79f8 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:56.136719 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c1be9-b6c5-473f-81ae-c313e1d846dd None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:56.153260 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:56.182770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7af50f-6bb6-4fd5-a146-f84f51483729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:56.183796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7af50f-6bb6-4fd5-a146-f84f51483729 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:56.184056 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7af50f-6bb6-4fd5-a146-f84f51483729 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:56.184325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7af50f-6bb6-4fd5-a146-f84f51483729 None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:56.185608 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7af50f-6bb6-4fd5-a146-f84f51483729 None admin] RBAC: Authorizing `identity:get_project(project_id=aad3af41314e48b4a48d335366f333ce)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:56.187155 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7af50f-6bb6-4fd5-a146-f84f51483729 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:56.197984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 108/225] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:48:55 2022] GET /identity/v3/projects/aad3af41314e48b4a48d335366f333ce => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:56.197984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7bffc-d99c-4bb2-9221-968a1c3b4ccd None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:56.197984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7bffc-d99c-4bb2-9221-968a1c3b4ccd None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:56.197984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7bffc-d99c-4bb2-9221-968a1c3b4ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:56.223742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-0ed7bffc-d99c-4bb2-9221-968a1c3b4ccd None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:56.225594 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-0ed7bffc-d99c-4bb2-9221-968a1c3b4ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:56.268917 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:56.275868 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b6fed-92a4-4c2d-b33a-f9c9e32b79f8 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:56.341359 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab66485-ebdc-449d-ab4b-0c112a61ab77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:56.343061 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab66485-ebdc-449d-ab4b-0c112a61ab77 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:56.345017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab66485-ebdc-449d-ab4b-0c112a61ab77 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:56.345017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab66485-ebdc-449d-ab4b-0c112a61ab77 None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:56.347349 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab66485-ebdc-449d-ab4b-0c112a61ab77 None admin] RBAC: Authorizing `identity:get_project(project_id=aad3af41314e48b4a48d335366f333ce)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:56.349731 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab66485-ebdc-449d-ab4b-0c112a61ab77 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:56.353827 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 118/226] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:48:56 2022] GET /identity/v3/projects/aad3af41314e48b4a48d335366f333ce => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:56.354172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab66485-ebdc-449d-ab4b-0c112a61ab77 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:56.356442 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 109/227] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:48:56.361981 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7bffc-d99c-4bb2-9221-968a1c3b4ccd None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:56.550337 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7bffc-d99c-4bb2-9221-968a1c3b4ccd None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:56.574602 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab66485-ebdc-449d-ab4b-0c112a61ab77 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:56.604936 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:56.638426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7e14c-dd5a-4690-8842-ba2e5e5e9396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:56.639786 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7e14c-dd5a-4690-8842-ba2e5e5e9396 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:56.640362 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7e14c-dd5a-4690-8842-ba2e5e5e9396 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:56.640904 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7e14c-dd5a-4690-8842-ba2e5e5e9396 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:56.642747 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7e14c-dd5a-4690-8842-ba2e5e5e9396 None admin] RBAC: Adding query filter params (role.id=b7f92100b08f4dc6b2b6866a2955d2ab, scope.project.id=aad3af41314e48b4a48d335366f333ce, user.id=d0b6e49f4c414f668e6e44be04eba19d) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:56.643357 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7e14c-dd5a-4690-8842-ba2e5e5e9396 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:56.646175 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7e14c-dd5a-4690-8842-ba2e5e5e9396 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:56.651058 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ee895-63cf-4d69-8f49-711ebfd28a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:56.652733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ee895-63cf-4d69-8f49-711ebfd28a7e None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:56.653287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ee895-63cf-4d69-8f49-711ebfd28a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:56.654006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ee895-63cf-4d69-8f49-711ebfd28a7e None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce/users/4b51fbf7a8cf4b3ab54456f31a074a6f/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:56.659173 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ee895-63cf-4d69-8f49-711ebfd28a7e None admin] RBAC: Authorizing `identity:create_grant(project_id=aad3af41314e48b4a48d335366f333ce, user_id=4b51fbf7a8cf4b3ab54456f31a074a6f, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:56.662310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ee895-63cf-4d69-8f49-711ebfd28a7e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:56.663341 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 110/228] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:48:56 2022] GET /identity/v3/role_assignments?user.id=d0b6e49f4c414f668e6e44be04eba19d&scope.project.id=aad3af41314e48b4a48d335366f333ce&role.id=b7f92100b08f4dc6b2b6866a2955d2ab => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:56.665684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7e14c-dd5a-4690-8842-ba2e5e5e9396 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:56.723093 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:56.827872 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 119/229] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:48:56 2022] PUT /identity/v3/projects/aad3af41314e48b4a48d335366f333ce/users/4b51fbf7a8cf4b3ab54456f31a074a6f/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:48:56.875075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7e14c-dd5a-4690-8842-ba2e5e5e9396 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:56.938572 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c642a03-46d1-409b-8e45-5cedd17a86c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:56.939480 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c642a03-46d1-409b-8e45-5cedd17a86c7 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:56.939763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c642a03-46d1-409b-8e45-5cedd17a86c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:56.940035 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c642a03-46d1-409b-8e45-5cedd17a86c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:56.960603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c642a03-46d1-409b-8e45-5cedd17a86c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:48:56.969746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-2c642a03-46d1-409b-8e45-5cedd17a86c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:48:56.969746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:48:56.969746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:48:56.969746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:48:56.969746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:48:56.969746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:48:56.969746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:48:56.969746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:48:56.969746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:48:56.969746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:48:56.969746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:48:56.969746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:48:56.969746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:48:56.969746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:48:56.969746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:48:56.969746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:48:56.971329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:48:56.971329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:48:56.971329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:48:56.971329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:48:56.971329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:48:56.971329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:48:56.971329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:48:56.971329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:48:56.971329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:48:56.971329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:48:56.971329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:48:56.971329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:48:56.971329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:48:56.971329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:48:56.971329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:48:56.971329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:48:56.973395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:48:56.973395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:48:56.973395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:48:56.973395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:48:56.973395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:48:56.973395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:48:56.973395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:48:56.973395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:48:56.973395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:48:56.973395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:48:56.973395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:48:56.973395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:48:56.973395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 111/230] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:48:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:48:56.984013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ee895-63cf-4d69-8f49-711ebfd28a7e None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:57.201846 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ee895-63cf-4d69-8f49-711ebfd28a7e None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:57.277847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc6545-b316-44b9-a9dc-6bd0564ad43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:57.279082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc6545-b316-44b9-a9dc-6bd0564ad43c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:57.279565 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc6545-b316-44b9-a9dc-6bd0564ad43c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:57.280026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc6545-b316-44b9-a9dc-6bd0564ad43c None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:57.281381 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc6545-b316-44b9-a9dc-6bd0564ad43c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:48:57.281931 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc6545-b316-44b9-a9dc-6bd0564ad43c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:57.284724 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc6545-b316-44b9-a9dc-6bd0564ad43c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:57.307162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 120/231] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:48:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:57.316050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c642a03-46d1-409b-8e45-5cedd17a86c7 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:57.467569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:57.492109 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c642a03-46d1-409b-8e45-5cedd17a86c7 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:57.537629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee18454-90f4-4608-b2f4-448a23e27553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:57.538514 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee18454-90f4-4608-b2f4-448a23e27553 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:57.538801 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee18454-90f4-4608-b2f4-448a23e27553 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:57.539121 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee18454-90f4-4608-b2f4-448a23e27553 None admin] PATH_INFO: `/v3/users/4b51fbf7a8cf4b3ab54456f31a074a6f` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:57.540589 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee18454-90f4-4608-b2f4-448a23e27553 None admin] RBAC: Authorizing `identity:get_user(user_id=4b51fbf7a8cf4b3ab54456f31a074a6f)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:57.542401 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee18454-90f4-4608-b2f4-448a23e27553 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:57.544502 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 112/232] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:48:57 2022] GET /identity/v3/users/4b51fbf7a8cf4b3ab54456f31a074a6f => generated 283 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:57.550259 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc6545-b316-44b9-a9dc-6bd0564ad43c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:57.636063 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:57.700510 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc6545-b316-44b9-a9dc-6bd0564ad43c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:57.739123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf4e98-bb02-4b8b-939e-68357ba4151e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:57.739901 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf4e98-bb02-4b8b-939e-68357ba4151e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:57.740187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf4e98-bb02-4b8b-939e-68357ba4151e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:57.740526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf4e98-bb02-4b8b-939e-68357ba4151e None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:57.741833 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf4e98-bb02-4b8b-939e-68357ba4151e None admin] RBAC: Authorizing `identity:get_project(project_id=aad3af41314e48b4a48d335366f333ce)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:57.743390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf4e98-bb02-4b8b-939e-68357ba4151e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:57.750412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 121/233] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:48:57 2022] GET /identity/v3/projects/aad3af41314e48b4a48d335366f333ce => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:48:57.756025 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee18454-90f4-4608-b2f4-448a23e27553 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:48:57.953550 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee18454-90f4-4608-b2f4-448a23e27553 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:48:58.030031 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac56a2-e422-4191-9abb-86286d828bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:48:58.031972 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac56a2-e422-4191-9abb-86286d828bab None admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:58.032740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac56a2-e422-4191-9abb-86286d828bab None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:58.033417 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac56a2-e422-4191-9abb-86286d828bab None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce/users/4b51fbf7a8cf4b3ab54456f31a074a6f/roles/af11cd5265d14ac7902b1fb3afcc4be7` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:58.037504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac56a2-e422-4191-9abb-86286d828bab None admin] RBAC: Authorizing `identity:create_grant(project_id=aad3af41314e48b4a48d335366f333ce, user_id=4b51fbf7a8cf4b3ab54456f31a074a6f, role_id=af11cd5265d14ac7902b1fb3afcc4be7)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:48:58.040870 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac56a2-e422-4191-9abb-86286d828bab None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:48:58.161852 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 113/234] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:48:57 2022] PUT /identity/v3/projects/aad3af41314e48b4a48d335366f333ce/users/4b51fbf7a8cf4b3ab54456f31a074a6f/roles/af11cd5265d14ac7902b1fb3afcc4be7 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:48:59.750420 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85bd5e-592a-41a2-9ea7-7ef4a531a905 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:59.751238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85bd5e-592a-41a2-9ea7-7ef4a531a905 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:59.751238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85bd5e-592a-41a2-9ea7-7ef4a531a905 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:59.752840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 122/235] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:48:59.759297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7328952-af24-4bba-8649-943e0f7583b7 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:59.760082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7328952-af24-4bba-8649-943e0f7583b7 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:59.760082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7328952-af24-4bba-8649-943e0f7583b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:59.788915 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-a7328952-af24-4bba-8649-943e0f7583b7 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:59.791163 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a7328952-af24-4bba-8649-943e0f7583b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:59.806766 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:48:59.889844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351808c9-364c-4c8a-9abb-20f714a3fa7a None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:59.889844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351808c9-364c-4c8a-9abb-20f714a3fa7a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:59.889844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351808c9-364c-4c8a-9abb-20f714a3fa7a None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:59.889844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 123/236] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:48:59.901032 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e387ca-35cd-43cb-93b0-fdd16c248e70 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:48:59.901032 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e387ca-35cd-43cb-93b0-fdd16c248e70 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:48:59.901032 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e387ca-35cd-43cb-93b0-fdd16c248e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:48:59.913409 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 114/237] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 14:48:59.942958 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-63e387ca-35cd-43cb-93b0-fdd16c248e70 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:48:59.944172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-63e387ca-35cd-43cb-93b0-fdd16c248e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:48:59.965590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7328952-af24-4bba-8649-943e0f7583b7 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:00.104989 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 124/238] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 14:49:00.107690 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07598b1b-b4ab-4508-b140-02cb1f2ca011 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:00.108272 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07598b1b-b4ab-4508-b140-02cb1f2ca011 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:00.108860 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07598b1b-b4ab-4508-b140-02cb1f2ca011 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:00.111829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 125/239] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:00.114835 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046ae6a-5413-4db3-80a6-10dc43bbb0ac None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:00.117488 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046ae6a-5413-4db3-80a6-10dc43bbb0ac None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:00.119548 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046ae6a-5413-4db3-80a6-10dc43bbb0ac None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:00.121323 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 126/240] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:00.125352 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9973f6-9ae4-4e19-a91d-de9d59d14849 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:00.126197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9973f6-9ae4-4e19-a91d-de9d59d14849 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:00.127011 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9973f6-9ae4-4e19-a91d-de9d59d14849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:00.154147 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7328952-af24-4bba-8649-943e0f7583b7 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:00.159171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-2d9973f6-9ae4-4e19-a91d-de9d59d14849 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:00.161373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2d9973f6-9ae4-4e19-a91d-de9d59d14849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:00.208096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a0850-3ebb-42d7-98d3-6775b14c6778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:00.209393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a0850-3ebb-42d7-98d3-6775b14c6778 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:00.209775 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a0850-3ebb-42d7-98d3-6775b14c6778 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:00.210192 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a0850-3ebb-42d7-98d3-6775b14c6778 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:00.213058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a0850-3ebb-42d7-98d3-6775b14c6778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:00.215296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a0850-3ebb-42d7-98d3-6775b14c6778 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:00.215666 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:00.222491 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 115/241] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:48:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:00.222491 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274d09a-2e01-4240-91bc-6ec01d310aa3 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:00.222491 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274d09a-2e01-4240-91bc-6ec01d310aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:00.222491 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274d09a-2e01-4240-91bc-6ec01d310aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:00.248240 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-b274d09a-2e01-4240-91bc-6ec01d310aa3 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:00.250229 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b274d09a-2e01-4240-91bc-6ec01d310aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:00.256269 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:00.284573 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 127/242] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:00.285815 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9973f6-9ae4-4e19-a91d-de9d59d14849 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:00.363720 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 116/243] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:00.368841 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274d09a-2e01-4240-91bc-6ec01d310aa3 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:00.445210 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9973f6-9ae4-4e19-a91d-de9d59d14849 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:00.487538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302ea69-72c2-4cd8-8f4c-72cc08e45c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:00.488353 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302ea69-72c2-4cd8-8f4c-72cc08e45c53 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:00.488633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302ea69-72c2-4cd8-8f4c-72cc08e45c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:00.488904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302ea69-72c2-4cd8-8f4c-72cc08e45c53 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:00.504187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274d09a-2e01-4240-91bc-6ec01d310aa3 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:00.506098 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3302ea69-72c2-4cd8-8f4c-72cc08e45c53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:00.516282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-3302ea69-72c2-4cd8-8f4c-72cc08e45c53 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 14:49:00.516282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:00.516282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:00.516282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:00.516282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:00.516282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:00.516282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:00.516282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:00.516282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:00.516282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:00.516282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:00.516282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:00.516282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:00.516282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:00.516282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:00.516282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:00.518002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:00.518002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:00.518002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:00.518002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:00.518002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:00.518002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:00.518002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:00.518002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:00.518002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:00.518002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:00.518002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:00.518002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:00.518002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:00.518002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:00.518002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:00.518002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:00.520861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:00.520861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:00.520861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:00.520861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:00.520861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:00.520861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:00.520861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:00.520861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:00.520861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:00.520861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:00.520861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 14:49:00.520861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:00.520861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 128/244] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:00.520861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302ea69-72c2-4cd8-8f4c-72cc08e45c53 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:00.554744 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:00.557596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66de8b6-34f5-4b5d-858b-90356836599e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:00.558436 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66de8b6-34f5-4b5d-858b-90356836599e None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:00.558929 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66de8b6-34f5-4b5d-858b-90356836599e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:00.559347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66de8b6-34f5-4b5d-858b-90356836599e None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:00.560452 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66de8b6-34f5-4b5d-858b-90356836599e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:00.562308 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66de8b6-34f5-4b5d-858b-90356836599e None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:00.575290 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:00.641610 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302ea69-72c2-4cd8-8f4c-72cc08e45c53 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:00.677861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653acdb-0b68-4772-85ff-2f624789b364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:00.678664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653acdb-0b68-4772-85ff-2f624789b364 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:00.678967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653acdb-0b68-4772-85ff-2f624789b364 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:00.679244 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653acdb-0b68-4772-85ff-2f624789b364 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:00.691566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2653acdb-0b68-4772-85ff-2f624789b364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:00.700053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-2653acdb-0b68-4772-85ff-2f624789b364 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:49:00.700053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:00.700053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:00.700053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:00.700053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:00.700053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:00.700053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:00.700053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:00.700053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:00.700053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:00.700053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:00.700053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:00.700053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:00.700053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:00.700053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:00.700053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:00.701726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:00.701726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:00.701726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:00.701726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:00.701726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:00.701726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:00.701726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:00.701726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:00.701726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:00.701726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:00.701726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:00.701726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:00.701726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:00.701726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:00.701726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:00.701726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:00.703277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:00.703277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:00.703277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:00.703277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:00.703277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:00.703277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:00.703277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:00.703277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:00.703277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:00.703277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:00.703277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:49:00.703277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:00.703277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 129/245] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:49:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:00.703277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653acdb-0b68-4772-85ff-2f624789b364 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:00.710416 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 117/246] 10.222.0.27 () {60 vars in 1132 bytes} [Sat Jul 16 14:49:00 2022] POST /identity/v3/users => generated 293 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:49:00.711735 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66de8b6-34f5-4b5d-858b-90356836599e None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:00.888531 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66de8b6-34f5-4b5d-858b-90356836599e None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:00.944904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:00.975290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653acdb-0b68-4772-85ff-2f624789b364 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:00.983789 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc38e3-32a4-46de-9e39-53d64499d94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:00.984497 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc38e3-32a4-46de-9e39-53d64499d94b None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:00.984775 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc38e3-32a4-46de-9e39-53d64499d94b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:00.985048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc38e3-32a4-46de-9e39-53d64499d94b None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:00.985820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbc38e3-32a4-46de-9e39-53d64499d94b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:00.986120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbc38e3-32a4-46de-9e39-53d64499d94b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:00.987704 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbc38e3-32a4-46de-9e39-53d64499d94b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:01.007823 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 118/247] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:49:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:01.009313 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc38e3-32a4-46de-9e39-53d64499d94b None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:01.051637 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685221f-2b63-4a2d-8f9a-703ec189dd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:01.053463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685221f-2b63-4a2d-8f9a-703ec189dd1b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:01.054167 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685221f-2b63-4a2d-8f9a-703ec189dd1b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:01.054617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685221f-2b63-4a2d-8f9a-703ec189dd1b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:01.076883 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685221f-2b63-4a2d-8f9a-703ec189dd1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:01.094770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-8685221f-2b63-4a2d-8f9a-703ec189dd1b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:01.094770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:01.094770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:01.094770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:01.094770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:01.094770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:01.094770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:01.094770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:01.094770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:01.094770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:01.094770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:01.094770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:01.094770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:01.094770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:01.094770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:01.094770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:01.095733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:01.095733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:01.095733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:01.095733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:01.095733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:01.095733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:01.095733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:01.095733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:01.095733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:01.095733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:01.095733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:01.095733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:01.095733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:01.095733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:01.095733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:01.095733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:01.096534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:01.096534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:01.096534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:01.096534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:01.096534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:01.096534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:01.096534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:01.096534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:01.096534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:01.096534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:01.096534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:01.096534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:01.097156 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 130/248] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:01.098148 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685221f-2b63-4a2d-8f9a-703ec189dd1b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:01.121585 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:01.180060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc38e3-32a4-46de-9e39-53d64499d94b None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:01.231723 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de08426-f381-4c24-b7c1-68ddc314d375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:01.232793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de08426-f381-4c24-b7c1-68ddc314d375 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:01.233304 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de08426-f381-4c24-b7c1-68ddc314d375 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:01.233779 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de08426-f381-4c24-b7c1-68ddc314d375 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:01.235464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de08426-f381-4c24-b7c1-68ddc314d375 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:01.235983 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de08426-f381-4c24-b7c1-68ddc314d375 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:01.237593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de08426-f381-4c24-b7c1-68ddc314d375 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:01.252499 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 119/249] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:01.253920 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de08426-f381-4c24-b7c1-68ddc314d375 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:01.286202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685221f-2b63-4a2d-8f9a-703ec189dd1b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:01.351161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a5b81-4692-4357-907c-f7acffb8f3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:01.353499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a5b81-4692-4357-907c-f7acffb8f3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:01.354658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a5b81-4692-4357-907c-f7acffb8f3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:01.355276 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a5b81-4692-4357-907c-f7acffb8f3f9 None admin] PATH_INFO: `/v3/users/42d10777035f4c66a26d7536f849b38f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:01.391555 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a5b81-4692-4357-907c-f7acffb8f3f9 None admin] RBAC: Authorizing `identity:get_user(user_id=42d10777035f4c66a26d7536f849b38f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:01.393424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a5b81-4692-4357-907c-f7acffb8f3f9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:01.395453 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 131/250] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:49:01 2022] GET /identity/v3/users/42d10777035f4c66a26d7536f849b38f => generated 297 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:01.396454 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a5b81-4692-4357-907c-f7acffb8f3f9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:01.404458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:01.473003 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de08426-f381-4c24-b7c1-68ddc314d375 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:01.507351 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:01.527521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513568b-73e6-483a-9141-3800417a464a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:01.528246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513568b-73e6-483a-9141-3800417a464a None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:01.528531 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513568b-73e6-483a-9141-3800417a464a None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:01.528838 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513568b-73e6-483a-9141-3800417a464a None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:01.529640 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513568b-73e6-483a-9141-3800417a464a None admin] RBAC: Adding query filter params (name=member) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:01.529948 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513568b-73e6-483a-9141-3800417a464a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:01.531523 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513568b-73e6-483a-9141-3800417a464a None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:01.543612 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 120/251] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:49:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:01.544268 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513568b-73e6-483a-9141-3800417a464a None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:01.653064 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a5b81-4692-4357-907c-f7acffb8f3f9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:01.726183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513568b-73e6-483a-9141-3800417a464a None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:01.735542 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5f0fd-ef12-4276-8911-9466e6f4409d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:01.737069 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5f0fd-ef12-4276-8911-9466e6f4409d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:01.737354 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5f0fd-ef12-4276-8911-9466e6f4409d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:01.737624 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5f0fd-ef12-4276-8911-9466e6f4409d None admin] PATH_INFO: `/v3/users/4b51fbf7a8cf4b3ab54456f31a074a6f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:01.739457 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5f0fd-ef12-4276-8911-9466e6f4409d None admin] RBAC: Authorizing `identity:get_user(user_id=4b51fbf7a8cf4b3ab54456f31a074a6f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:01.740942 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5f0fd-ef12-4276-8911-9466e6f4409d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:01.744705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 132/252] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:49:01 2022] GET /identity/v3/users/4b51fbf7a8cf4b3ab54456f31a074a6f => generated 283 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:01.745749 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5f0fd-ef12-4276-8911-9466e6f4409d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:01.781502 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39432d83-2457-495e-9e9c-4a7a62dea42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:01.782510 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39432d83-2457-495e-9e9c-4a7a62dea42d None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:01.782801 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39432d83-2457-495e-9e9c-4a7a62dea42d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:01.783685 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39432d83-2457-495e-9e9c-4a7a62dea42d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:01.846696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:01.879025 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:01.889616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5f0fd-ef12-4276-8911-9466e6f4409d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:01.906378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39432d83-2457-495e-9e9c-4a7a62dea42d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:01.909421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39432d83-2457-495e-9e9c-4a7a62dea42d None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:01.950099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3dc4a9-f563-4c8c-9cea-b5237ab42c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:01.952364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dc4a9-f563-4c8c-9cea-b5237ab42c7a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:01.952364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dc4a9-f563-4c8c-9cea-b5237ab42c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:01.952899 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3dc4a9-f563-4c8c-9cea-b5237ab42c7a None admin] PATH_INFO: `/v3/users/d0b6e49f4c414f668e6e44be04eba19d` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:01.955178 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3dc4a9-f563-4c8c-9cea-b5237ab42c7a None admin] RBAC: Authorizing `identity:get_user(user_id=d0b6e49f4c414f668e6e44be04eba19d)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:01.957984 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3dc4a9-f563-4c8c-9cea-b5237ab42c7a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:01.966424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 133/253] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:49:01 2022] GET /identity/v3/users/d0b6e49f4c414f668e6e44be04eba19d => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:01.966424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d496849-cd61-4823-8146-ed62746464c6 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:01.966424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d496849-cd61-4823-8146-ed62746464c6 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:01.966424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d496849-cd61-4823-8146-ed62746464c6 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:01.968320 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 134/254] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:01.969727 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d496849-cd61-4823-8146-ed62746464c6 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:02.042592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 121/255] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:49:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:49:02.044421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39432d83-2457-495e-9e9c-4a7a62dea42d None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:02.126185 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d496849-cd61-4823-8146-ed62746464c6 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:02.175445 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5e184-6d44-4843-bbbb-e8a6377cbacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:02.176838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5e184-6d44-4843-bbbb-e8a6377cbacd None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:02.177326 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5e184-6d44-4843-bbbb-e8a6377cbacd None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:02.177784 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5e184-6d44-4843-bbbb-e8a6377cbacd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:02.197798 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:02.217135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39432d83-2457-495e-9e9c-4a7a62dea42d None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:02.224873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:02.283692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5e184-6d44-4843-bbbb-e8a6377cbacd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:02.285538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5e184-6d44-4843-bbbb-e8a6377cbacd None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:02.292100 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb58fe3-1dfe-433d-a0a1-c1e373b8b688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:02.294208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb58fe3-1dfe-433d-a0a1-c1e373b8b688 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:02.294568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb58fe3-1dfe-433d-a0a1-c1e373b8b688 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:02.295231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb58fe3-1dfe-433d-a0a1-c1e373b8b688 None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:02.298623 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb58fe3-1dfe-433d-a0a1-c1e373b8b688 None admin] RBAC: Authorizing `identity:get_project(project_id=aad3af41314e48b4a48d335366f333ce)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:02.302964 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb58fe3-1dfe-433d-a0a1-c1e373b8b688 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:02.309106 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 122/256] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:02 2022] GET /identity/v3/projects/aad3af41314e48b4a48d335366f333ce => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:02.314266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89280ce5-8af6-4bec-b5db-98f5a852d936 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:02.314266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89280ce5-8af6-4bec-b5db-98f5a852d936 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:02.314266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89280ce5-8af6-4bec-b5db-98f5a852d936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:02.346498 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-89280ce5-8af6-4bec-b5db-98f5a852d936 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:02.348047 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-89280ce5-8af6-4bec-b5db-98f5a852d936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:02.434297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 123/257] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:02.440158 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89280ce5-8af6-4bec-b5db-98f5a852d936 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:02.501447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 135/258] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:49:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:49:02.502684 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5e184-6d44-4843-bbbb-e8a6377cbacd None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:02.570839 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:02.589588 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:02.613009 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89280ce5-8af6-4bec-b5db-98f5a852d936 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:02.654106 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11543d-c775-4556-a459-c56dd7618bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:02.655388 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11543d-c775-4556-a459-c56dd7618bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:02.655929 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11543d-c775-4556-a459-c56dd7618bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:02.656438 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11543d-c775-4556-a459-c56dd7618bd9 None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:02.658097 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11543d-c775-4556-a459-c56dd7618bd9 None admin] RBAC: Authorizing `identity:get_project(project_id=aad3af41314e48b4a48d335366f333ce)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:02.659940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11543d-c775-4556-a459-c56dd7618bd9 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:02.662262 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 124/259] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:02 2022] GET /identity/v3/projects/aad3af41314e48b4a48d335366f333ce => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:02.663312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11543d-c775-4556-a459-c56dd7618bd9 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:02.693969 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5e184-6d44-4843-bbbb-e8a6377cbacd None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:02.754533 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1cd6d-ba64-468b-9fd9-6133a159b855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:02.756873 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1cd6d-ba64-468b-9fd9-6133a159b855 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:02.757362 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1cd6d-ba64-468b-9fd9-6133a159b855 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:02.758171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1cd6d-ba64-468b-9fd9-6133a159b855 None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce/users/d0b6e49f4c414f668e6e44be04eba19d/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:02.763075 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1cd6d-ba64-468b-9fd9-6133a159b855 None admin] RBAC: Authorizing `identity:create_grant(project_id=aad3af41314e48b4a48d335366f333ce, user_id=d0b6e49f4c414f668e6e44be04eba19d, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:02.766881 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1cd6d-ba64-468b-9fd9-6133a159b855 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:02.779252 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11543d-c775-4556-a459-c56dd7618bd9 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:02.821830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfac27-b026-4979-a609-2703587f6993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:02.822712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfac27-b026-4979-a609-2703587f6993 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:02.823080 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfac27-b026-4979-a609-2703587f6993 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:02.823442 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfac27-b026-4979-a609-2703587f6993 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:02.836481 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdfac27-b026-4979-a609-2703587f6993 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:02.845642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 136/260] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:49:02 2022] PUT /identity/v3/projects/aad3af41314e48b4a48d335366f333ce/users/d0b6e49f4c414f668e6e44be04eba19d/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:49:02.845898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-bbdfac27-b026-4979-a609-2703587f6993 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:49:02.845898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:02.845898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:02.845898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:02.845898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:02.845898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:02.845898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:02.845898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:02.845898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:02.845898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:02.845898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:02.845898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:02.845898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:02.845898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:02.845898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:02.845898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:02.847653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:02.847653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:02.847653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:02.847653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:02.847653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:02.847653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:02.847653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:02.847653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:02.847653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:02.847653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:02.847653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:02.847653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:02.847653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:02.847653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:02.847653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:02.847653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:02.849318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:02.849318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:02.849318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:02.849318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:02.849318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:02.849318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:02.849318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:02.849318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:02.849318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:02.849318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:02.849318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:49:02.849318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:02.854934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 125/261] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:49:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:02.854934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfac27-b026-4979-a609-2703587f6993 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:02.857902 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1cd6d-ba64-468b-9fd9-6133a159b855 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:02.883790 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:03.105061 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:03.113624 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1cd6d-ba64-468b-9fd9-6133a159b855 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:03.162866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfac27-b026-4979-a609-2703587f6993 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:03.205689 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461bc91-b3c2-4bd7-9d76-49d7dabd432e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:03.207769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461bc91-b3c2-4bd7-9d76-49d7dabd432e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:03.214405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461bc91-b3c2-4bd7-9d76-49d7dabd432e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:03.214405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461bc91-b3c2-4bd7-9d76-49d7dabd432e None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:03.214405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461bc91-b3c2-4bd7-9d76-49d7dabd432e None admin] RBAC: Authorizing `identity:get_project(project_id=aad3af41314e48b4a48d335366f333ce)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:03.216603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461bc91-b3c2-4bd7-9d76-49d7dabd432e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:03.219101 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528be161-f406-43fc-ba3a-26667e1505ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:03.219799 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528be161-f406-43fc-ba3a-26667e1505ae None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:03.220076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528be161-f406-43fc-ba3a-26667e1505ae None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:03.220357 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528be161-f406-43fc-ba3a-26667e1505ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:03.220670 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 137/262] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:02 2022] GET /identity/v3/projects/aad3af41314e48b4a48d335366f333ce => generated 302 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:03.221235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528be161-f406-43fc-ba3a-26667e1505ae None admin] RBAC: Adding query filter params (role.id=7f966f669d82448daddfe911e2955ea2, scope.project.id=aad3af41314e48b4a48d335366f333ce, user.id=42d10777035f4c66a26d7536f849b38f) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:03.221550 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528be161-f406-43fc-ba3a-26667e1505ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:03.221900 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461bc91-b3c2-4bd7-9d76-49d7dabd432e None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:03.223091 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528be161-f406-43fc-ba3a-26667e1505ae None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:03.230838 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 126/263] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:49:02 2022] GET /identity/v3/role_assignments?user.id=42d10777035f4c66a26d7536f849b38f&scope.project.id=aad3af41314e48b4a48d335366f333ce&role.id=7f966f669d82448daddfe911e2955ea2 => generated 259 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:03.242431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528be161-f406-43fc-ba3a-26667e1505ae None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:03.339904 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:03.381141 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528be161-f406-43fc-ba3a-26667e1505ae None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:03.467441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461bc91-b3c2-4bd7-9d76-49d7dabd432e None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:03.493573 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4454b-7108-43c1-974e-4913f77e5bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:03.494632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4454b-7108-43c1-974e-4913f77e5bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:03.495048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4454b-7108-43c1-974e-4913f77e5bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:03.495429 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4454b-7108-43c1-974e-4913f77e5bf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:03.496425 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4454b-7108-43c1-974e-4913f77e5bf3 None admin] RBAC: Adding query filter params (role.id=0efeff46573247f3b9f1e2dec2968672, scope.project.id=aad3af41314e48b4a48d335366f333ce, user.id=4b51fbf7a8cf4b3ab54456f31a074a6f) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:03.496866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4454b-7108-43c1-974e-4913f77e5bf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:03.498483 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4454b-7108-43c1-974e-4913f77e5bf3 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:03.519210 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 127/264] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:49:03 2022] GET /identity/v3/role_assignments?user.id=4b51fbf7a8cf4b3ab54456f31a074a6f&scope.project.id=aad3af41314e48b4a48d335366f333ce&role.id=0efeff46573247f3b9f1e2dec2968672 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:03.553175 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24d8ae-db2c-4679-9c26-9fc08029ee24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:03.554459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24d8ae-db2c-4679-9c26-9fc08029ee24 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:03.555006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24d8ae-db2c-4679-9c26-9fc08029ee24 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:03.555544 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24d8ae-db2c-4679-9c26-9fc08029ee24 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:03.556498 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356e327-702b-4083-b3eb-e814a562ac5e None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:03.556838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a24d8ae-db2c-4679-9c26-9fc08029ee24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:03.556992 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356e327-702b-4083-b3eb-e814a562ac5e None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:03.557240 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a24d8ae-db2c-4679-9c26-9fc08029ee24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:03.557397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356e327-702b-4083-b3eb-e814a562ac5e None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:03.558574 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a24d8ae-db2c-4679-9c26-9fc08029ee24 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:03.558837 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 128/265] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:03.564737 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec567223-411e-4dd2-9e68-a0abcf50335f None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:03.565039 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec567223-411e-4dd2-9e68-a0abcf50335f None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:03.565298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec567223-411e-4dd2-9e68-a0abcf50335f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:03.569179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 138/266] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:49:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:03.577608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24d8ae-db2c-4679-9c26-9fc08029ee24 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:03.608089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-ec567223-411e-4dd2-9e68-a0abcf50335f None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:03.609380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ec567223-411e-4dd2-9e68-a0abcf50335f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:03.630391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:03.751311 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24d8ae-db2c-4679-9c26-9fc08029ee24 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:03.777797 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 129/267] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:03.799134 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23eaa8d-fca9-441b-9077-b4d4da4c2d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:03.800041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23eaa8d-fca9-441b-9077-b4d4da4c2d44 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:03.800320 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23eaa8d-fca9-441b-9077-b4d4da4c2d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:03.800592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23eaa8d-fca9-441b-9077-b4d4da4c2d44 None admin] PATH_INFO: `/v3/users/4b51fbf7a8cf4b3ab54456f31a074a6f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:03.801933 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23eaa8d-fca9-441b-9077-b4d4da4c2d44 None admin] RBAC: Authorizing `identity:get_user(user_id=4b51fbf7a8cf4b3ab54456f31a074a6f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:03.803508 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23eaa8d-fca9-441b-9077-b4d4da4c2d44 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:03.805639 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 139/268] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:49:03 2022] GET /identity/v3/users/4b51fbf7a8cf4b3ab54456f31a074a6f => generated 283 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:49:03.811611 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec567223-411e-4dd2-9e68-a0abcf50335f None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:03.812255 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23eaa8d-fca9-441b-9077-b4d4da4c2d44 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:03.944644 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:03.958939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23eaa8d-fca9-441b-9077-b4d4da4c2d44 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:04.023379 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:04.036127 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a0a43-bb70-4100-80e3-164ec78e4876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:04.038042 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a0a43-bb70-4100-80e3-164ec78e4876 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:04.038807 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a0a43-bb70-4100-80e3-164ec78e4876 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:04.039571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a0a43-bb70-4100-80e3-164ec78e4876 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:04.044669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a0a43-bb70-4100-80e3-164ec78e4876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:04.047779 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a0a43-bb70-4100-80e3-164ec78e4876 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:04.051669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 140/269] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:49:04.057914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a0a43-bb70-4100-80e3-164ec78e4876 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:04.117134 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec567223-411e-4dd2-9e68-a0abcf50335f None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:04.201519 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a590f0-68e7-4ba3-b311-df9501babf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:04.202182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a590f0-68e7-4ba3-b311-df9501babf0e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:04.202466 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a590f0-68e7-4ba3-b311-df9501babf0e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:04.202742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a590f0-68e7-4ba3-b311-df9501babf0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:04.288154 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a590f0-68e7-4ba3-b311-df9501babf0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:04.290324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a590f0-68e7-4ba3-b311-df9501babf0e None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:04.315098 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a0a43-bb70-4100-80e3-164ec78e4876 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:04.366779 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:04.409048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 130/270] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:49:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:49:04.414726 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a590f0-68e7-4ba3-b311-df9501babf0e None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:04.467773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ab3ce4-3b61-4666-b367-4d1f4ad00269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:04.468474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab3ce4-3b61-4666-b367-4d1f4ad00269 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:04.468754 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab3ce4-3b61-4666-b367-4d1f4ad00269 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:04.469022 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab3ce4-3b61-4666-b367-4d1f4ad00269 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:04.469768 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab3ce4-3b61-4666-b367-4d1f4ad00269 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:04.471296 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ab3ce4-3b61-4666-b367-4d1f4ad00269 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:04.532217 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a590f0-68e7-4ba3-b311-df9501babf0e None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:04.572068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b2048-57a9-49fc-a3ef-1c51925f9268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:04.572878 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b2048-57a9-49fc-a3ef-1c51925f9268 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:04.573157 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b2048-57a9-49fc-a3ef-1c51925f9268 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:04.573432 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b2048-57a9-49fc-a3ef-1c51925f9268 None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:04.574705 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b2048-57a9-49fc-a3ef-1c51925f9268 None admin] RBAC: Authorizing `identity:get_project(project_id=aad3af41314e48b4a48d335366f333ce)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:04.576240 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b2048-57a9-49fc-a3ef-1c51925f9268 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:04.583242 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 131/271] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:04 2022] GET /identity/v3/projects/aad3af41314e48b4a48d335366f333ce => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:04.583242 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13e64d-88db-4a15-a16f-0c913b988c13 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:04.583242 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13e64d-88db-4a15-a16f-0c913b988c13 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:04.583242 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13e64d-88db-4a15-a16f-0c913b988c13 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:04.583242 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 132/272] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:04.584090 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13e64d-88db-4a15-a16f-0c913b988c13 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:04.600982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 141/273] 10.222.0.27 () {60 vars in 1132 bytes} [Sat Jul 16 14:49:04 2022] POST /identity/v3/users => generated 293 bytes in 544 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 14:49:04.604049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dff01a-c735-47e3-9ac7-f6c7c38e57cb None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:04.604636 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dff01a-c735-47e3-9ac7-f6c7c38e57cb None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:04.605106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dff01a-c735-47e3-9ac7-f6c7c38e57cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:04.652803 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-13dff01a-c735-47e3-9ac7-f6c7c38e57cb None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:04.654110 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-13dff01a-c735-47e3-9ac7-f6c7c38e57cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:04.667318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:04.696153 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:04.827676 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 142/274] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:04.829821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13e64d-88db-4a15-a16f-0c913b988c13 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:04.860865 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dff01a-c735-47e3-9ac7-f6c7c38e57cb None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:04.909282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292ddb9-4129-4423-96ea-29387e54b67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:04.909849 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292ddb9-4129-4423-96ea-29387e54b67a None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:04.910129 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292ddb9-4129-4423-96ea-29387e54b67a None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:04.910398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292ddb9-4129-4423-96ea-29387e54b67a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:04.911283 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1292ddb9-4129-4423-96ea-29387e54b67a None admin] RBAC: Adding query filter params (role.id=af11cd5265d14ac7902b1fb3afcc4be7, scope.project.id=aad3af41314e48b4a48d335366f333ce, user.id=4b51fbf7a8cf4b3ab54456f31a074a6f) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:04.911591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1292ddb9-4129-4423-96ea-29387e54b67a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:04.913079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1292ddb9-4129-4423-96ea-29387e54b67a None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:04.921431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 133/275] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:49:04 2022] GET /identity/v3/role_assignments?user.id=4b51fbf7a8cf4b3ab54456f31a074a6f&scope.project.id=aad3af41314e48b4a48d335366f333ce&role.id=af11cd5265d14ac7902b1fb3afcc4be7 => generated 608 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:05.060496 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dff01a-c735-47e3-9ac7-f6c7c38e57cb None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:05.133123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e682ae-1765-412d-9ef8-ef7f20a17889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:05.134512 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e682ae-1765-412d-9ef8-ef7f20a17889 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:05.135049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e682ae-1765-412d-9ef8-ef7f20a17889 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:05.135609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e682ae-1765-412d-9ef8-ef7f20a17889 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:05.157125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e682ae-1765-412d-9ef8-ef7f20a17889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:05.172470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-36e682ae-1765-412d-9ef8-ef7f20a17889 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:05.172470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:05.172470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:05.172470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:05.172470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:05.172470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:05.172470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:05.172470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:05.172470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:05.172470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:05.172470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:05.172470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:05.172470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:05.172470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:05.172470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:05.172470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:05.173433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:05.173433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:05.173433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:05.173433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:05.173433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:05.173433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:05.173433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:05.173433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:05.173433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:05.173433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:05.173433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:05.173433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:05.173433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:05.173433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:05.173433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:05.173433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:05.174238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:05.174238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:05.174238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:05.174238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:05.174238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:05.174238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:05.174238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:05.174238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:05.174238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:05.174238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:05.174238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:05.174238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:05.175549 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 143/276] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:05.181416 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292ddb9-4129-4423-96ea-29387e54b67a None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:05.326539 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:05.329368 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292ddb9-4129-4423-96ea-29387e54b67a None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:05.377259 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296e4dc-3cb9-4cfa-9fea-c2a15d1e2b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:05.378308 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296e4dc-3cb9-4cfa-9fea-c2a15d1e2b83 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:05.378683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296e4dc-3cb9-4cfa-9fea-c2a15d1e2b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:05.380103 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296e4dc-3cb9-4cfa-9fea-c2a15d1e2b83 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:05.380887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296e4dc-3cb9-4cfa-9fea-c2a15d1e2b83 None admin] RBAC: Adding query filter params (name=member) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:05.381192 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296e4dc-3cb9-4cfa-9fea-c2a15d1e2b83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:05.382614 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296e4dc-3cb9-4cfa-9fea-c2a15d1e2b83 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:05.401823 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 134/277] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:49:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:05.409260 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e682ae-1765-412d-9ef8-ef7f20a17889 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:05.435795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:05.587808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e682ae-1765-412d-9ef8-ef7f20a17889 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:05.635737 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b2014-caf2-4fea-aa14-e5f61c12a148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:05.636647 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b2014-caf2-4fea-aa14-e5f61c12a148 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:05.636920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b2014-caf2-4fea-aa14-e5f61c12a148 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:05.637188 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b2014-caf2-4fea-aa14-e5f61c12a148 None admin] PATH_INFO: `/v3/users/af43d1544d9d48a98ae368798adac95a` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:05.659864 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b2014-caf2-4fea-aa14-e5f61c12a148 None admin] RBAC: Authorizing `identity:get_user(user_id=af43d1544d9d48a98ae368798adac95a)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:05.661275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b2014-caf2-4fea-aa14-e5f61c12a148 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:05.663082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 144/278] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:49:05 2022] GET /identity/v3/users/af43d1544d9d48a98ae368798adac95a => generated 293 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:05.670672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296e4dc-3cb9-4cfa-9fea-c2a15d1e2b83 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:05.737359 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08277283-b126-4a49-b08d-5c66c0bcb371 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:05.741882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08277283-b126-4a49-b08d-5c66c0bcb371 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:05.741882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08277283-b126-4a49-b08d-5c66c0bcb371 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:05.741882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 145/279] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:49:05.748574 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68eeb8-c066-4af5-8cac-f2121cbda564 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:05.748858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68eeb8-c066-4af5-8cac-f2121cbda564 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:05.749131 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68eeb8-c066-4af5-8cac-f2121cbda564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:05.784527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-8d68eeb8-c066-4af5-8cac-f2121cbda564 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:05.786549 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-8d68eeb8-c066-4af5-8cac-f2121cbda564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:06.043514 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296e4dc-3cb9-4cfa-9fea-c2a15d1e2b83 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:06.056829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:06.064125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 146/280] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 14:49:06.102457 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c0db6-38eb-4f4d-9b48-f6f0617aaf8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:06.103162 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c0db6-38eb-4f4d-9b48-f6f0617aaf8c None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:06.103438 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c0db6-38eb-4f4d-9b48-f6f0617aaf8c None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:06.103707 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c0db6-38eb-4f4d-9b48-f6f0617aaf8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:06.104553 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c0db6-38eb-4f4d-9b48-f6f0617aaf8c None admin] RBAC: Adding query filter params (role.id=b7f92100b08f4dc6b2b6866a2955d2ab, user.id=af43d1544d9d48a98ae368798adac95a, scope.system=all) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:06.104860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c0db6-38eb-4f4d-9b48-f6f0617aaf8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:06.106337 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c0db6-38eb-4f4d-9b48-f6f0617aaf8c None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:06.113830 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68eeb8-c066-4af5-8cac-f2121cbda564 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:06.128071 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 135/281] 10.222.0.27 () {56 vars in 1308 bytes} [Sat Jul 16 14:49:05 2022] GET /identity/v3/role_assignments?user.id=af43d1544d9d48a98ae368798adac95a&scope.system=all&role.id=b7f92100b08f4dc6b2b6866a2955d2ab => generated 226 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:06.296964 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d68eeb8-c066-4af5-8cac-f2121cbda564 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:06.349115 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fe82b-1cea-4ee2-8942-fce05d1324de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:06.349945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fe82b-1cea-4ee2-8942-fce05d1324de None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:06.350227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fe82b-1cea-4ee2-8942-fce05d1324de None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:06.350500 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fe82b-1cea-4ee2-8942-fce05d1324de None admin] PATH_INFO: `/v3/roles/member` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:06.366921 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86fe82b-1cea-4ee2-8942-fce05d1324de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:06.375450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-c86fe82b-1cea-4ee2-8942-fce05d1324de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:06.375450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:06.375450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:06.375450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:06.375450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:06.375450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:06.375450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:06.375450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:06.375450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:06.375450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:06.375450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:06.375450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:06.375450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:06.375450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:06.375450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:06.375450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:06.376466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:06.376466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:06.376466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:06.376466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:06.376466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:06.376466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:06.376466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:06.376466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:06.376466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:06.376466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:06.376466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:06.376466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:06.376466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:06.376466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:06.376466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:06.376466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:06.377254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:06.377254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:06.377254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:06.377254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:06.377254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:06.377254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:06.377254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:06.377254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:06.377254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:06.377254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:06.377254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:06.377254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:06.377254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 147/282] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:49:06.385339 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c0db6-38eb-4f4d-9b48-f6f0617aaf8c None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:06.465915 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:06.645256 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c0db6-38eb-4f4d-9b48-f6f0617aaf8c None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:06.719068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf78097-2359-4ded-81b0-8b5c7e28dded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:06.720172 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf78097-2359-4ded-81b0-8b5c7e28dded None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:06.720690 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf78097-2359-4ded-81b0-8b5c7e28dded None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:06.721188 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf78097-2359-4ded-81b0-8b5c7e28dded None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:06.722430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf78097-2359-4ded-81b0-8b5c7e28dded None admin] RBAC: Adding query filter params (name=member) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:06.723058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf78097-2359-4ded-81b0-8b5c7e28dded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:06.725703 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf78097-2359-4ded-81b0-8b5c7e28dded None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:06.740555 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e82275-a3cc-40ba-9119-c64ea49cb36e None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:06.741226 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e82275-a3cc-40ba-9119-c64ea49cb36e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:06.741226 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e82275-a3cc-40ba-9119-c64ea49cb36e None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:06.742493 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 148/283] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:06.743714 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 136/284] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:49:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:06.750818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e82275-a3cc-40ba-9119-c64ea49cb36e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:06.766357 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb1ba9-4f22-45f3-8ee2-d9be117d1ffc None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:06.766357 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb1ba9-4f22-45f3-8ee2-d9be117d1ffc None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:06.766619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb1ba9-4f22-45f3-8ee2-d9be117d1ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:06.797099 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-0abb1ba9-4f22-45f3-8ee2-d9be117d1ffc None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:06.806926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-0abb1ba9-4f22-45f3-8ee2-d9be117d1ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:06.866281 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:06.999624 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 137/285] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:07.011959 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e82275-a3cc-40ba-9119-c64ea49cb36e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:07.041829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb1ba9-4f22-45f3-8ee2-d9be117d1ffc None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:07.108124 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:07.120888 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d8365-c4b1-46b1-ae37-3a9cd11079d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:07.121786 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d8365-c4b1-46b1-ae37-3a9cd11079d9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:07.122064 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d8365-c4b1-46b1-ae37-3a9cd11079d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:07.122333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d8365-c4b1-46b1-ae37-3a9cd11079d9 None admin] PATH_INFO: `/v3/users/d0b6e49f4c414f668e6e44be04eba19d` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:07.127785 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d8365-c4b1-46b1-ae37-3a9cd11079d9 None admin] RBAC: Authorizing `identity:get_user(user_id=d0b6e49f4c414f668e6e44be04eba19d)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:07.129306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d8365-c4b1-46b1-ae37-3a9cd11079d9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:07.131410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 149/286] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:49:06 2022] GET /identity/v3/users/d0b6e49f4c414f668e6e44be04eba19d => generated 297 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:07.138017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d8365-c4b1-46b1-ae37-3a9cd11079d9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:07.242933 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb1ba9-4f22-45f3-8ee2-d9be117d1ffc None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:07.281934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71acc413-cf09-4570-8de3-829e0bf1004b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:07.282755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acc413-cf09-4570-8de3-829e0bf1004b None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:07.283089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acc413-cf09-4570-8de3-829e0bf1004b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:07.283371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acc413-cf09-4570-8de3-829e0bf1004b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:07.296150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71acc413-cf09-4570-8de3-829e0bf1004b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:07.310336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-71acc413-cf09-4570-8de3-829e0bf1004b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 14:49:07.310336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:07.310336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:07.310336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:07.310336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:07.310336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:07.310336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:07.310336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:07.310336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:07.310336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:07.310336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:07.310336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:07.310336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:07.310336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:07.310336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:07.310336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:07.311353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:07.311353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:07.311353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:07.311353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:07.311353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:07.311353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:07.311353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:07.311353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:07.311353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:07.311353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:07.311353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:07.311353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:07.311353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:07.311353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:07.311353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:07.311353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:07.312590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:07.312590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:07.312590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:07.312590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:07.312590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:07.312590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:07.312590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:07.312590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:07.312590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:07.312590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:07.312590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 14:49:07.312590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:07.313416 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 138/287] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:49:07.319203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71acc413-cf09-4570-8de3-829e0bf1004b None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:07.463280 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:07.464941 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d8365-c4b1-46b1-ae37-3a9cd11079d9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:07.479834 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71acc413-cf09-4570-8de3-829e0bf1004b None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:07.506740 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:07.528768 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25895268-5b8b-45b9-8c7a-f674e1baf1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:07.529449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25895268-5b8b-45b9-8c7a-f674e1baf1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:07.529732 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25895268-5b8b-45b9-8c7a-f674e1baf1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:07.530007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25895268-5b8b-45b9-8c7a-f674e1baf1d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:07.562482 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac23ed-a9a5-4430-afac-21650bb7eb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:07.563210 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac23ed-a9a5-4430-afac-21650bb7eb4e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:07.563492 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac23ed-a9a5-4430-afac-21650bb7eb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:07.563764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac23ed-a9a5-4430-afac-21650bb7eb4e None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:07.564542 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac23ed-a9a5-4430-afac-21650bb7eb4e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:07.564846 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac23ed-a9a5-4430-afac-21650bb7eb4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:07.566257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac23ed-a9a5-4430-afac-21650bb7eb4e None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:07.585800 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 139/288] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:49:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:49:07.592861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac23ed-a9a5-4430-afac-21650bb7eb4e None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:07.679529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25895268-5b8b-45b9-8c7a-f674e1baf1d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:07.681732 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25895268-5b8b-45b9-8c7a-f674e1baf1d7 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:07.746360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac23ed-a9a5-4430-afac-21650bb7eb4e None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:07.786127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd09665-005d-40d8-8cd2-5f5c221fe4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:07.787058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd09665-005d-40d8-8cd2-5f5c221fe4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:07.787341 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd09665-005d-40d8-8cd2-5f5c221fe4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:07.787613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd09665-005d-40d8-8cd2-5f5c221fe4f6 None admin] PATH_INFO: `/v3/users/42d10777035f4c66a26d7536f849b38f` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:07.788954 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd09665-005d-40d8-8cd2-5f5c221fe4f6 None admin] RBAC: Authorizing `identity:get_user(user_id=42d10777035f4c66a26d7536f849b38f)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:07.791699 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd09665-005d-40d8-8cd2-5f5c221fe4f6 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:07.793941 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 140/289] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:49:07 2022] GET /identity/v3/users/42d10777035f4c66a26d7536f849b38f => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:07.795306 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ac042-b4e2-48d4-a929-27832518080e None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:07.795582 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ac042-b4e2-48d4-a929-27832518080e None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:07.795853 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ac042-b4e2-48d4-a929-27832518080e None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:07.797158 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 141/290] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:07.802052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ac042-b4e2-48d4-a929-27832518080e None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:07.814276 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 150/291] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:49:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:49:07.815723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9348a-10a5-4be5-99bd-01fc45689d48 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:07.816014 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9348a-10a5-4be5-99bd-01fc45689d48 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:07.816292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9348a-10a5-4be5-99bd-01fc45689d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:07.818459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:07.838088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-dcc9348a-10a5-4be5-99bd-01fc45689d48 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:07.839649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-dcc9348a-10a5-4be5-99bd-01fc45689d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:07.868166 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:07.926809 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 151/292] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:07.935052 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9348a-10a5-4be5-99bd-01fc45689d48 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:07.960730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ac042-b4e2-48d4-a929-27832518080e None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:08.015553 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165706f-0fce-4dd7-9ca2-2a7d5b42220f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:08.016699 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165706f-0fce-4dd7-9ca2-2a7d5b42220f None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:08.017148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165706f-0fce-4dd7-9ca2-2a7d5b42220f None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:08.017560 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165706f-0fce-4dd7-9ca2-2a7d5b42220f None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:08.019118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165706f-0fce-4dd7-9ca2-2a7d5b42220f None admin] RBAC: Authorizing `identity:get_project(project_id=aad3af41314e48b4a48d335366f333ce)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:08.020988 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165706f-0fce-4dd7-9ca2-2a7d5b42220f None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:08.023200 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 142/293] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:07 2022] GET /identity/v3/projects/aad3af41314e48b4a48d335366f333ce => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:08.024866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165706f-0fce-4dd7-9ca2-2a7d5b42220f None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:08.053847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9348a-10a5-4be5-99bd-01fc45689d48 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:08.092432 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af15ae-926a-4344-aa84-5f22d9026ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:08.093310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af15ae-926a-4344-aa84-5f22d9026ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:08.093591 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af15ae-926a-4344-aa84-5f22d9026ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:08.093860 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af15ae-926a-4344-aa84-5f22d9026ab4 None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:08.095144 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af15ae-926a-4344-aa84-5f22d9026ab4 None admin] RBAC: Authorizing `identity:get_project(project_id=aad3af41314e48b4a48d335366f333ce)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:08.096603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af15ae-926a-4344-aa84-5f22d9026ab4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:08.098367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 152/294] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:07 2022] GET /identity/v3/projects/aad3af41314e48b4a48d335366f333ce => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:08.099680 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552d989-fe02-4614-9e0e-55bcb677b704 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:08.099962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552d989-fe02-4614-9e0e-55bcb677b704 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:08.100237 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552d989-fe02-4614-9e0e-55bcb677b704 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:08.106543 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 153/295] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:08.106543 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552d989-fe02-4614-9e0e-55bcb677b704 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:08.124370 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:08.133190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:08.153060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165706f-0fce-4dd7-9ca2-2a7d5b42220f None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:08.208069 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55946947-0ee7-4bba-a7cc-c4340a0db5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:08.209680 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55946947-0ee7-4bba-a7cc-c4340a0db5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:08.210366 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55946947-0ee7-4bba-a7cc-c4340a0db5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:08.210753 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55946947-0ee7-4bba-a7cc-c4340a0db5f1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:08.229904 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55946947-0ee7-4bba-a7cc-c4340a0db5f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:08.244504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-55946947-0ee7-4bba-a7cc-c4340a0db5f1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 14:49:08.244504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:08.244504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:08.244504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:08.244504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:08.244504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:08.244504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:08.244504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:08.244504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:08.244504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:08.244504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:08.244504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:08.244504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:08.244504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:08.244504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:08.244504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:08.246315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:08.246315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:08.246315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:08.246315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:08.246315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:08.246315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:08.246315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:08.246315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:08.246315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:08.246315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:08.246315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:08.246315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:08.246315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:08.246315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:08.246315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:08.246315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:08.248601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:08.248601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:08.248601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:08.248601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:08.248601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:08.248601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:08.248601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:08.248601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:08.248601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:08.248601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:08.248601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 14:49:08.248601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:08.248601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 143/296] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:08.248601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55946947-0ee7-4bba-a7cc-c4340a0db5f1 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:08.267119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552d989-fe02-4614-9e0e-55bcb677b704 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:08.317698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4785b-c9cd-4b5e-ba1e-9103f218e774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:08.318586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4785b-c9cd-4b5e-ba1e-9103f218e774 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:08.319043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4785b-c9cd-4b5e-ba1e-9103f218e774 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:08.319429 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4785b-c9cd-4b5e-ba1e-9103f218e774 None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce/users/42d10777035f4c66a26d7536f849b38f/roles/7f966f669d82448daddfe911e2955ea2` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:08.323571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4785b-c9cd-4b5e-ba1e-9103f218e774 None admin] RBAC: Authorizing `identity:create_grant(project_id=aad3af41314e48b4a48d335366f333ce, user_id=42d10777035f4c66a26d7536f849b38f, role_id=7f966f669d82448daddfe911e2955ea2)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:08.326219 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4785b-c9cd-4b5e-ba1e-9103f218e774 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:08.407347 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 154/297] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:49:08 2022] PUT /identity/v3/projects/aad3af41314e48b4a48d335366f333ce/users/42d10777035f4c66a26d7536f849b38f/roles/7f966f669d82448daddfe911e2955ea2 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:49:08.408935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55946947-0ee7-4bba-a7cc-c4340a0db5f1 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:08.409368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92baeddb-07db-49e1-9668-2f44d3034e0c None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:08.409858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92baeddb-07db-49e1-9668-2f44d3034e0c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:08.410333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92baeddb-07db-49e1-9668-2f44d3034e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:08.431879 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-92baeddb-07db-49e1-9668-2f44d3034e0c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:08.433672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-92baeddb-07db-49e1-9668-2f44d3034e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:08.481108 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf650c4-0099-47a9-bd2c-8f1063d2a0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:08.482677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf650c4-0099-47a9-bd2c-8f1063d2a0ec None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:08.483249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf650c4-0099-47a9-bd2c-8f1063d2a0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:08.483834 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf650c4-0099-47a9-bd2c-8f1063d2a0ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:08.485023 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf650c4-0099-47a9-bd2c-8f1063d2a0ec None admin] RBAC: Adding query filter params (role.id=b7f92100b08f4dc6b2b6866a2955d2ab, scope.project.id=aad3af41314e48b4a48d335366f333ce, user.id=d0b6e49f4c414f668e6e44be04eba19d) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:08.485609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf650c4-0099-47a9-bd2c-8f1063d2a0ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:08.487455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf650c4-0099-47a9-bd2c-8f1063d2a0ec None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:08.492320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:08.498292 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 144/298] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:49:08 2022] GET /identity/v3/role_assignments?user.id=d0b6e49f4c414f668e6e44be04eba19d&scope.project.id=aad3af41314e48b4a48d335366f333ce&role.id=b7f92100b08f4dc6b2b6866a2955d2ab => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:08.499617 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf650c4-0099-47a9-bd2c-8f1063d2a0ec None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:08.587958 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 155/299] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:08.590623 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2dc25-887a-4cc5-aadf-1aa6e6e2f005 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:08.591783 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2dc25-887a-4cc5-aadf-1aa6e6e2f005 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:08.592311 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2dc25-887a-4cc5-aadf-1aa6e6e2f005 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:08.594187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 156/300] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:08.608823 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebb8e3-c5a7-49c7-b068-6673fe84fb26 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:08.609323 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebb8e3-c5a7-49c7-b068-6673fe84fb26 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:08.609813 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebb8e3-c5a7-49c7-b068-6673fe84fb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:08.640746 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-36ebb8e3-c5a7-49c7-b068-6673fe84fb26 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:08.643020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-36ebb8e3-c5a7-49c7-b068-6673fe84fb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:08.661613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:08.674309 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf650c4-0099-47a9-bd2c-8f1063d2a0ec None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:08.751043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 157/301] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 14:49:08.752427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ebb8e3-c5a7-49c7-b068-6673fe84fb26 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:08.753082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c6a31-831b-44f4-ab7f-8c07628633d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:08.753721 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c6a31-831b-44f4-ab7f-8c07628633d2 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:08.754007 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c6a31-831b-44f4-ab7f-8c07628633d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:08.754287 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c6a31-831b-44f4-ab7f-8c07628633d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:08.755097 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c6a31-831b-44f4-ab7f-8c07628633d2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:08.755413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c6a31-831b-44f4-ab7f-8c07628633d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:08.756828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c6a31-831b-44f4-ab7f-8c07628633d2 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:08.772156 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 145/302] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:49:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:08.781304 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c6a31-831b-44f4-ab7f-8c07628633d2 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:08.907346 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:08.943763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ebb8e3-c5a7-49c7-b068-6673fe84fb26 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:08.991920 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c6a31-831b-44f4-ab7f-8c07628633d2 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:08.992652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cb307-b14b-4f33-99eb-aa6543d82889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:08.994137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cb307-b14b-4f33-99eb-aa6543d82889 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:08.994633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cb307-b14b-4f33-99eb-aa6543d82889 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:08.995303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cb307-b14b-4f33-99eb-aa6543d82889 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:08.996663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cb307-b14b-4f33-99eb-aa6543d82889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:08.999086 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cb307-b14b-4f33-99eb-aa6543d82889 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:09.001347 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 158/303] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:09.002226 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cb307-b14b-4f33-99eb-aa6543d82889 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:09.030482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:09.053596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d09129-b805-4219-849c-9bb8b60cc6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:09.055183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d09129-b805-4219-849c-9bb8b60cc6fa None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:09.056330 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d09129-b805-4219-849c-9bb8b60cc6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:09.056330 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d09129-b805-4219-849c-9bb8b60cc6fa None admin] PATH_INFO: `/v3/users/e3b7400e8bf44a7f8874c897cb2e1980` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:09.088211 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d09129-b805-4219-849c-9bb8b60cc6fa None admin] RBAC: Authorizing `identity:get_user(user_id=e3b7400e8bf44a7f8874c897cb2e1980)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:09.092051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d09129-b805-4219-849c-9bb8b60cc6fa None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:09.096518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 146/304] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:49:08 2022] GET /identity/v3/users/e3b7400e8bf44a7f8874c897cb2e1980 => generated 293 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:09.097943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d09129-b805-4219-849c-9bb8b60cc6fa None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:09.185141 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cb307-b14b-4f33-99eb-aa6543d82889 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:09.238307 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0c429-7bbc-40d8-b335-712257347369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:09.240063 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0c429-7bbc-40d8-b335-712257347369 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:09.240857 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0c429-7bbc-40d8-b335-712257347369 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:09.241659 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0c429-7bbc-40d8-b335-712257347369 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:09.259637 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d09129-b805-4219-849c-9bb8b60cc6fa None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:09.260080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0c429-7bbc-40d8-b335-712257347369 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:09.269863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-bac0c429-7bbc-40d8-b335-712257347369 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:09.269863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:09.269863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:09.269863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:09.269863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:09.269863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:09.269863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:09.269863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:09.269863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:09.269863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:09.269863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:09.269863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:09.269863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:09.269863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:09.269863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:09.269863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:09.271937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:09.271937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:09.271937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:09.271937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:09.271937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:09.271937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:09.271937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:09.271937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:09.271937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:09.271937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:09.271937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:09.271937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:09.271937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:09.271937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:09.271937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:09.271937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:09.273477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:09.273477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:09.273477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:09.273477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:09.273477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:09.273477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:09.273477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:09.273477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:09.273477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:09.273477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:09.273477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:09.273477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:09.273477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 159/305] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:09.273477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0c429-7bbc-40d8-b335-712257347369 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:09.297513 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:09.323925 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4610b13-4e97-4773-82a3-28f9ecdbb9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:09.325064 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4610b13-4e97-4773-82a3-28f9ecdbb9dc None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:09.325587 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4610b13-4e97-4773-82a3-28f9ecdbb9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:09.326057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4610b13-4e97-4773-82a3-28f9ecdbb9dc None admin] PATH_INFO: `/v3/groups` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:09.327338 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4610b13-4e97-4773-82a3-28f9ecdbb9dc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:09.330151 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4610b13-4e97-4773-82a3-28f9ecdbb9dc None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:09.390620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:09.410400 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 147/306] 10.222.0.27 () {60 vars in 1134 bytes} [Sat Jul 16 14:49:09 2022] POST /identity/v3/groups => generated 229 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:49:09.411709 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4610b13-4e97-4773-82a3-28f9ecdbb9dc None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:09.466621 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0c429-7bbc-40d8-b335-712257347369 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:09.550985 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a9df4-8e92-42a7-a2d2-8ebaba5eee9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:09.552904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a9df4-8e92-42a7-a2d2-8ebaba5eee9f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:09.553684 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a9df4-8e92-42a7-a2d2-8ebaba5eee9f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:09.554675 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a9df4-8e92-42a7-a2d2-8ebaba5eee9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:09.559054 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a9df4-8e92-42a7-a2d2-8ebaba5eee9f None admin] RBAC: Adding query filter params (role.id=7f966f669d82448daddfe911e2955ea2, user.id=e3b7400e8bf44a7f8874c897cb2e1980, scope.system=all) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:09.559997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a9df4-8e92-42a7-a2d2-8ebaba5eee9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:09.562946 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a9df4-8e92-42a7-a2d2-8ebaba5eee9f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:09.583210 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 160/307] 10.222.0.27 () {56 vars in 1308 bytes} [Sat Jul 16 14:49:09 2022] GET /identity/v3/role_assignments?user.id=e3b7400e8bf44a7f8874c897cb2e1980&scope.system=all&role.id=7f966f669d82448daddfe911e2955ea2 => generated 226 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:09.622583 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4610b13-4e97-4773-82a3-28f9ecdbb9dc None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:09.692050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83d5fe-0196-423e-8d53-7fb332becfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:09.693216 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83d5fe-0196-423e-8d53-7fb332becfab None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:09.693709 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83d5fe-0196-423e-8d53-7fb332becfab None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:09.694203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83d5fe-0196-423e-8d53-7fb332becfab None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:09.695726 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83d5fe-0196-423e-8d53-7fb332becfab None admin] RBAC: Adding query filter params (name=member) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:09.696814 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83d5fe-0196-423e-8d53-7fb332becfab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:09.699764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83d5fe-0196-423e-8d53-7fb332becfab None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:09.719043 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 148/308] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:49:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:09.725362 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a9df4-8e92-42a7-a2d2-8ebaba5eee9f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:09.909442 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a9df4-8e92-42a7-a2d2-8ebaba5eee9f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:09.980732 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51dff4-e9c1-4f8d-b03b-f80b2044cadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:09.982186 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51dff4-e9c1-4f8d-b03b-f80b2044cadb None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:09.982683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51dff4-e9c1-4f8d-b03b-f80b2044cadb None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:09.983199 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51dff4-e9c1-4f8d-b03b-f80b2044cadb None admin] PATH_INFO: `/v3/users/af43d1544d9d48a98ae368798adac95a` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:09.985671 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c51dff4-e9c1-4f8d-b03b-f80b2044cadb None admin] RBAC: Authorizing `identity:get_user(user_id=af43d1544d9d48a98ae368798adac95a)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:09.988463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c51dff4-e9c1-4f8d-b03b-f80b2044cadb None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:09.992257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 161/309] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:49:09 2022] GET /identity/v3/users/af43d1544d9d48a98ae368798adac95a => generated 293 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:09.998049 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83d5fe-0196-423e-8d53-7fb332becfab None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:10.073749 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:10.187989 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83d5fe-0196-423e-8d53-7fb332becfab None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:10.257518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b47d8a-997a-4dca-b034-4f05b20f7120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:10.258996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b47d8a-997a-4dca-b034-4f05b20f7120 None admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:10.259620 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b47d8a-997a-4dca-b034-4f05b20f7120 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:10.260361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b47d8a-997a-4dca-b034-4f05b20f7120 None admin] PATH_INFO: `/v3/system/users/af43d1544d9d48a98ae368798adac95a/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:10.263180 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b47d8a-997a-4dca-b034-4f05b20f7120 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=af43d1544d9d48a98ae368798adac95a, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:10.265998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b47d8a-997a-4dca-b034-4f05b20f7120 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:10.351272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 149/310] 10.222.0.27 () {58 vars in 1334 bytes} [Sat Jul 16 14:49:09 2022] PUT /identity/v3/system/users/af43d1544d9d48a98ae368798adac95a/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 14:49:11.636041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a67af-d600-4771-8c60-8159f7e5ad1a None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:11.636589 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a67af-d600-4771-8c60-8159f7e5ad1a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:11.636589 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58a67af-d600-4771-8c60-8159f7e5ad1a None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:11.637983 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 162/311] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:11.647462 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc58af2-1772-4d54-ad66-005e825fa195 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:11.648949 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc58af2-1772-4d54-ad66-005e825fa195 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:11.650355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc58af2-1772-4d54-ad66-005e825fa195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:11.681008 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-9fc58af2-1772-4d54-ad66-005e825fa195 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:11.683670 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-9fc58af2-1772-4d54-ad66-005e825fa195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:11.818216 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 150/312] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:11.848714 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b9b52-69cb-45d2-ac84-55d434df8e7d None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:11.850880 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b9b52-69cb-45d2-ac84-55d434df8e7d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:11.850880 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b9b52-69cb-45d2-ac84-55d434df8e7d None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:11.850880 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 163/313] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:11.857455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dbea3-8bba-4f32-b023-d08b3f6291c0 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:11.858098 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dbea3-8bba-4f32-b023-d08b3f6291c0 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:11.858581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809dbea3-8bba-4f32-b023-d08b3f6291c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:11.860172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b9b52-69cb-45d2-ac84-55d434df8e7d None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:11.869218 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:11.877209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-809dbea3-8bba-4f32-b023-d08b3f6291c0 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:11.880547 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-809dbea3-8bba-4f32-b023-d08b3f6291c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:11.918145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:11.971208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 151/314] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:11.972701 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e66b7-4290-4df3-8879-313ccae11a7a None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:11.973050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e66b7-4290-4df3-8879-313ccae11a7a None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:11.973615 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e66b7-4290-4df3-8879-313ccae11a7a None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:11.975249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 152/315] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:11.984743 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d5c38-8bf1-49ef-9642-29dfdc28604d None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:11.985181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d5c38-8bf1-49ef-9642-29dfdc28604d None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:11.985724 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d5c38-8bf1-49ef-9642-29dfdc28604d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:12.010498 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-d51d5c38-8bf1-49ef-9642-29dfdc28604d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:12.013079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-d51d5c38-8bf1-49ef-9642-29dfdc28604d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:12.023221 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b9b52-69cb-45d2-ac84-55d434df8e7d None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:12.089906 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1aae96-8330-499d-ba0c-3a87d1309383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:12.091639 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1aae96-8330-499d-ba0c-3a87d1309383 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:12.092133 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1aae96-8330-499d-ba0c-3a87d1309383 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:12.092615 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1aae96-8330-499d-ba0c-3a87d1309383 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:12.114995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 153/316] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:12.114995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d5c38-8bf1-49ef-9642-29dfdc28604d None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:12.115404 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1aae96-8330-499d-ba0c-3a87d1309383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:12.129267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-8a1aae96-8330-499d-ba0c-3a87d1309383 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 14:49:12.129267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:12.129267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:12.129267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:12.129267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:12.129267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:12.129267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:12.129267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:12.129267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:12.129267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:12.129267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:12.129267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:12.129267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:12.129267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:12.129267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:12.129267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:12.131275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:12.131275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:12.131275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:12.131275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:12.131275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:12.131275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:12.131275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:12.131275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:12.131275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:12.131275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:12.131275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:12.131275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:12.131275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:12.131275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:12.131275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:12.131275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:12.133024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:12.133024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:12.133024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:12.133024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:12.133024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:12.133024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:12.133024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:12.133024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:12.133024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:12.133024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:12.133024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 14:49:12.133024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:12.145293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 164/317] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:12.147153 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1aae96-8330-499d-ba0c-3a87d1309383 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:12.265549 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:12.277422 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:12.285997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d5c38-8bf1-49ef-9642-29dfdc28604d None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:12.329043 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ed1a8-7b24-4f2d-b8b8-7e0395aac991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:12.329982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ed1a8-7b24-4f2d-b8b8-7e0395aac991 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:12.330283 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ed1a8-7b24-4f2d-b8b8-7e0395aac991 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:12.330534 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ed1a8-7b24-4f2d-b8b8-7e0395aac991 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:12.338540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1aae96-8330-499d-ba0c-3a87d1309383 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:12.349949 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ed1a8-7b24-4f2d-b8b8-7e0395aac991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:12.366901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-793ed1a8-7b24-4f2d-b8b8-7e0395aac991 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 14:49:12.366901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:12.366901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:12.366901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:12.366901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:12.366901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:12.366901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:12.366901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:12.366901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:12.366901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:12.366901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:12.366901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:12.366901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:12.366901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:12.366901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:12.366901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:12.368448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:12.368448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:12.368448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:12.368448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:12.368448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:12.368448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:12.368448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:12.368448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:12.368448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:12.368448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:12.368448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:12.368448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:12.368448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:12.368448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:12.368448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:12.368448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:12.369427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:12.369427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:12.369427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:12.369427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:12.369427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:12.369427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:12.369427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:12.369427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:12.369427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:12.369427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:12.369427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 14:49:12.369427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:12.375000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 154/318] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:12.375000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ed1a8-7b24-4f2d-b8b8-7e0395aac991 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:12.407666 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600ea58-b384-4a83-badd-86c44d7ee027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:12.409166 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600ea58-b384-4a83-badd-86c44d7ee027 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:12.409922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600ea58-b384-4a83-badd-86c44d7ee027 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:12.410402 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600ea58-b384-4a83-badd-86c44d7ee027 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:12.411876 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600ea58-b384-4a83-badd-86c44d7ee027 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:12.412424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600ea58-b384-4a83-badd-86c44d7ee027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:12.415665 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600ea58-b384-4a83-badd-86c44d7ee027 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:12.437644 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 165/319] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:49:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:12.437644 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600ea58-b384-4a83-badd-86c44d7ee027 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:12.538261 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ed1a8-7b24-4f2d-b8b8-7e0395aac991 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:12.578867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600ea58-b384-4a83-badd-86c44d7ee027 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:12.580017 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067767a2-5a41-4c24-8e5c-d1b9eed29127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:12.580852 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067767a2-5a41-4c24-8e5c-d1b9eed29127 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:12.581137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067767a2-5a41-4c24-8e5c-d1b9eed29127 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:12.581411 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067767a2-5a41-4c24-8e5c-d1b9eed29127 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:12.582744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067767a2-5a41-4c24-8e5c-d1b9eed29127 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:12.584447 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067767a2-5a41-4c24-8e5c-d1b9eed29127 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:12.586235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 155/320] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:12.588548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067767a2-5a41-4c24-8e5c-d1b9eed29127 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:12.623740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:12.624377 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:12.638157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1001adda-bb7d-492c-a922-5cf0d4bcb163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:12.639261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001adda-bb7d-492c-a922-5cf0d4bcb163 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:12.639728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001adda-bb7d-492c-a922-5cf0d4bcb163 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:12.640138 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001adda-bb7d-492c-a922-5cf0d4bcb163 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:12.641031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1001adda-bb7d-492c-a922-5cf0d4bcb163 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:12.641428 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1001adda-bb7d-492c-a922-5cf0d4bcb163 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:12.642985 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1001adda-bb7d-492c-a922-5cf0d4bcb163 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:12.656605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 166/321] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:49:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:12.659303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41168f-5855-4569-a011-e570eea50846 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:12.660363 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41168f-5855-4569-a011-e570eea50846 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:12.667666 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41168f-5855-4569-a011-e570eea50846 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:12.667666 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 167/322] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:12.667666 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41168f-5855-4569-a011-e570eea50846 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:12.725113 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067767a2-5a41-4c24-8e5c-d1b9eed29127 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:12.769220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bbe2c-2b31-4205-96c7-a695eb7184b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:12.770611 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bbe2c-2b31-4205-96c7-a695eb7184b5 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:12.771103 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bbe2c-2b31-4205-96c7-a695eb7184b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:12.771551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bbe2c-2b31-4205-96c7-a695eb7184b5 None admin] PATH_INFO: `/v3/users/42d10777035f4c66a26d7536f849b38f` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:12.773314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91bbe2c-2b31-4205-96c7-a695eb7184b5 None admin] RBAC: Authorizing `identity:get_user(user_id=42d10777035f4c66a26d7536f849b38f)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:12.776425 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91bbe2c-2b31-4205-96c7-a695eb7184b5 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:12.779331 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 156/323] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:49:12 2022] GET /identity/v3/users/42d10777035f4c66a26d7536f849b38f => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:12.780299 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bbe2c-2b31-4205-96c7-a695eb7184b5 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:12.817059 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41168f-5855-4569-a011-e570eea50846 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:12.859998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e991f9-6159-4782-9551-0dc5c627206c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:12.860855 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e991f9-6159-4782-9551-0dc5c627206c None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:12.861205 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e991f9-6159-4782-9551-0dc5c627206c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:12.861547 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e991f9-6159-4782-9551-0dc5c627206c None admin] PATH_INFO: `/v3/groups` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:12.862370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e991f9-6159-4782-9551-0dc5c627206c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:12.864024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e991f9-6159-4782-9551-0dc5c627206c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:12.891228 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:12.903202 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bbe2c-2b31-4205-96c7-a695eb7184b5 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:12.942175 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25856a-2797-45c0-b2e2-46dad4c7fd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:12.943167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25856a-2797-45c0-b2e2-46dad4c7fd91 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:12.943451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25856a-2797-45c0-b2e2-46dad4c7fd91 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:12.947134 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25856a-2797-45c0-b2e2-46dad4c7fd91 None admin] PATH_INFO: `/v3/users/e3b7400e8bf44a7f8874c897cb2e1980` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:12.947134 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25856a-2797-45c0-b2e2-46dad4c7fd91 None admin] RBAC: Authorizing `identity:get_user(user_id=e3b7400e8bf44a7f8874c897cb2e1980)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:12.947539 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25856a-2797-45c0-b2e2-46dad4c7fd91 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:12.949245 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 157/324] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:49:12 2022] GET /identity/v3/users/e3b7400e8bf44a7f8874c897cb2e1980 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:12.950760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f984007-eaba-4361-b58f-014b257d1e8b None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:12.951255 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f984007-eaba-4361-b58f-014b257d1e8b None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:12.951400 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f984007-eaba-4361-b58f-014b257d1e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:12.959090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 168/325] 10.222.0.27 () {60 vars in 1134 bytes} [Sat Jul 16 14:49:12 2022] POST /identity/v3/groups => generated 226 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:49:12.960247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e991f9-6159-4782-9551-0dc5c627206c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:12.978311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-5f984007-eaba-4361-b58f-014b257d1e8b None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:12.981817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5f984007-eaba-4361-b58f-014b257d1e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:13.052609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:13.075399 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 158/326] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:13.076098 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f984007-eaba-4361-b58f-014b257d1e8b None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:13.146343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e991f9-6159-4782-9551-0dc5c627206c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:13.210210 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8eaf0-c105-407b-884c-9caaeee1fdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:13.210903 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8eaf0-c105-407b-884c-9caaeee1fdc6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:13.211203 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8eaf0-c105-407b-884c-9caaeee1fdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:13.211514 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8eaf0-c105-407b-884c-9caaeee1fdc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:13.244475 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:13.262393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f984007-eaba-4361-b58f-014b257d1e8b None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:13.312677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11e036-9d3b-454a-b440-665a5459879b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:13.313642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11e036-9d3b-454a-b440-665a5459879b None admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:13.313923 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11e036-9d3b-454a-b440-665a5459879b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:13.314200 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11e036-9d3b-454a-b440-665a5459879b None admin] PATH_INFO: `/v3/system/users/e3b7400e8bf44a7f8874c897cb2e1980/roles/7f966f669d82448daddfe911e2955ea2` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:13.315825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11e036-9d3b-454a-b440-665a5459879b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e3b7400e8bf44a7f8874c897cb2e1980, role_id=7f966f669d82448daddfe911e2955ea2)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:13.317490 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11e036-9d3b-454a-b440-665a5459879b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:13.336506 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8eaf0-c105-407b-884c-9caaeee1fdc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:13.338097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d8eaf0-c105-407b-884c-9caaeee1fdc6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:13.384736 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 159/327] 10.222.0.27 () {58 vars in 1334 bytes} [Sat Jul 16 14:49:13 2022] PUT /identity/v3/system/users/e3b7400e8bf44a7f8874c897cb2e1980/roles/7f966f669d82448daddfe911e2955ea2 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:49:13.386079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11e036-9d3b-454a-b440-665a5459879b None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:13.420998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:13.530670 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 169/328] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:49:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:49:13.542959 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8eaf0-c105-407b-884c-9caaeee1fdc6 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:13.610962 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11e036-9d3b-454a-b440-665a5459879b None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:13.665978 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093706a-19b8-4c09-bc3b-9ca74b3e6ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:13.666990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093706a-19b8-4c09-bc3b-9ca74b3e6ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:13.667357 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093706a-19b8-4c09-bc3b-9ca74b3e6ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:13.667630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093706a-19b8-4c09-bc3b-9ca74b3e6ab1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:13.680866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8093706a-19b8-4c09-bc3b-9ca74b3e6ab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:13.689105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-8093706a-19b8-4c09-bc3b-9ca74b3e6ab1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:13.689105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:13.689105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:13.689105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:13.689105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:13.689105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:13.689105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:13.689105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:13.689105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:13.689105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:13.689105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:13.689105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:13.689105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:13.689105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:13.689105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:13.689105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:13.690285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:13.690285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:13.690285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:13.690285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:13.690285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:13.690285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:13.690285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:13.690285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:13.690285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:13.690285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:13.690285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:13.690285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:13.690285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:13.690285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:13.690285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:13.690285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:13.691321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:13.691321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:13.691321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:13.691321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:13.691321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:13.691321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:13.691321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:13.691321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:13.691321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:13.691321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:13.691321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:13.691321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:13.691321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 160/329] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:13.699474 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093706a-19b8-4c09-bc3b-9ca74b3e6ab1 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:13.717002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8eaf0-c105-407b-884c-9caaeee1fdc6 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:13.726676 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:13.776712 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118350a4-f0d6-44b5-9ae1-ee8772d831ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:13.777674 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118350a4-f0d6-44b5-9ae1-ee8772d831ff None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:13.777956 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118350a4-f0d6-44b5-9ae1-ee8772d831ff None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:13.778226 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118350a4-f0d6-44b5-9ae1-ee8772d831ff None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:13.781949 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118350a4-f0d6-44b5-9ae1-ee8772d831ff None admin] RBAC: Authorizing `identity:get_project(project_id=aad3af41314e48b4a48d335366f333ce)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:13.784742 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118350a4-f0d6-44b5-9ae1-ee8772d831ff None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:13.788767 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 170/330] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:13 2022] GET /identity/v3/projects/aad3af41314e48b4a48d335366f333ce => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:13.797994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118350a4-f0d6-44b5-9ae1-ee8772d831ff None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:13.834557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:13.871529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093706a-19b8-4c09-bc3b-9ca74b3e6ab1 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:13.913368 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e4344-b5e2-4fc7-8f59-93df648e41bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:13.914656 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e4344-b5e2-4fc7-8f59-93df648e41bc None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:13.915690 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e4344-b5e2-4fc7-8f59-93df648e41bc None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:13.916916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e4344-b5e2-4fc7-8f59-93df648e41bc None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:13.917953 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e4344-b5e2-4fc7-8f59-93df648e41bc None admin] RBAC: Adding query filter params (name=member) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:13.918488 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e4344-b5e2-4fc7-8f59-93df648e41bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:13.920159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e4344-b5e2-4fc7-8f59-93df648e41bc None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:13.930660 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 161/331] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:49:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:49:13.937143 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e4344-b5e2-4fc7-8f59-93df648e41bc None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:14.009856 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118350a4-f0d6-44b5-9ae1-ee8772d831ff None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:14.027260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:14.076543 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e4344-b5e2-4fc7-8f59-93df648e41bc None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:14.089843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d306de-9666-47af-9237-274962941ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:14.091038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d306de-9666-47af-9237-274962941ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:14.091533 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d306de-9666-47af-9237-274962941ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:14.092010 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d306de-9666-47af-9237-274962941ae5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:14.093804 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d306de-9666-47af-9237-274962941ae5 None admin] RBAC: Adding query filter params (role.id=7f966f669d82448daddfe911e2955ea2, scope.project.id=aad3af41314e48b4a48d335366f333ce, user.id=42d10777035f4c66a26d7536f849b38f) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:14.094340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d306de-9666-47af-9237-274962941ae5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:14.097093 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d306de-9666-47af-9237-274962941ae5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:14.109520 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 171/332] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:49:13 2022] GET /identity/v3/role_assignments?user.id=42d10777035f4c66a26d7536f849b38f&scope.project.id=aad3af41314e48b4a48d335366f333ce&role.id=7f966f669d82448daddfe911e2955ea2 => generated 608 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:14.158192 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7b8c9-c6c0-4234-85d3-0951ceea4b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:14.159674 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7b8c9-c6c0-4234-85d3-0951ceea4b19 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:14.160164 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7b8c9-c6c0-4234-85d3-0951ceea4b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:14.160629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7b8c9-c6c0-4234-85d3-0951ceea4b19 None admin] PATH_INFO: `/v3/users/af43d1544d9d48a98ae368798adac95a` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:14.162990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7b8c9-c6c0-4234-85d3-0951ceea4b19 None admin] RBAC: Authorizing `identity:get_user(user_id=af43d1544d9d48a98ae368798adac95a)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:14.166393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7b8c9-c6c0-4234-85d3-0951ceea4b19 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:14.170201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 162/333] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:49:13 2022] GET /identity/v3/users/af43d1544d9d48a98ae368798adac95a => generated 293 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:14.179904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d306de-9666-47af-9237-274962941ae5 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:14.362391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:14.403770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d306de-9666-47af-9237-274962941ae5 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:14.485707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d2762-a5ac-4aba-acce-2a2c9750d51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:14.486751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d2762-a5ac-4aba-acce-2a2c9750d51f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:14.487274 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d2762-a5ac-4aba-acce-2a2c9750d51f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:14.487750 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d2762-a5ac-4aba-acce-2a2c9750d51f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:14.489416 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d2762-a5ac-4aba-acce-2a2c9750d51f None admin] RBAC: Adding query filter params (role.id=b7f92100b08f4dc6b2b6866a2955d2ab, user.id=af43d1544d9d48a98ae368798adac95a, scope.system=all) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:14.490081 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d2762-a5ac-4aba-acce-2a2c9750d51f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:14.493161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d2762-a5ac-4aba-acce-2a2c9750d51f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:14.518568 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 172/334] 10.222.0.27 () {56 vars in 1308 bytes} [Sat Jul 16 14:49:14 2022] GET /identity/v3/role_assignments?user.id=af43d1544d9d48a98ae368798adac95a&scope.system=all&role.id=b7f92100b08f4dc6b2b6866a2955d2ab => generated 510 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:16.065873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28d953-4f2c-457c-bfb5-9add98345190 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:16.066508 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28d953-4f2c-457c-bfb5-9add98345190 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:16.066508 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28d953-4f2c-457c-bfb5-9add98345190 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:16.067870 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 163/335] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:16.075790 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e91ac-d1f2-4f78-a9d8-04a4145ee73a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:16.076548 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e91ac-d1f2-4f78-a9d8-04a4145ee73a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:16.076548 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e91ac-d1f2-4f78-a9d8-04a4145ee73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:16.100815 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d02e91ac-d1f2-4f78-a9d8-04a4145ee73a None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:16.102079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d02e91ac-d1f2-4f78-a9d8-04a4145ee73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:16.199193 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 173/336] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 14:49:16.243464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28d953-4f2c-457c-bfb5-9add98345190 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:16.379823 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28d953-4f2c-457c-bfb5-9add98345190 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:16.424912 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546e4d6-89e4-4c99-9185-789464bb8f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:16.425724 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546e4d6-89e4-4c99-9185-789464bb8f96 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:16.426008 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546e4d6-89e4-4c99-9185-789464bb8f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:16.426284 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546e4d6-89e4-4c99-9185-789464bb8f96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:16.441801 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546e4d6-89e4-4c99-9185-789464bb8f96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:16.450421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-4546e4d6-89e4-4c99-9185-789464bb8f96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:16.450421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:16.450421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:16.450421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:16.450421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:16.450421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:16.450421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:16.450421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:16.450421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:16.450421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:16.450421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:16.450421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:16.450421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:16.450421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:16.450421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:16.450421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:16.452764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:16.452764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:16.452764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:16.452764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:16.452764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:16.452764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:16.452764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:16.452764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:16.452764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:16.452764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:16.452764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:16.452764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:16.452764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:16.452764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:16.452764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:16.452764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:16.454380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:16.454380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:16.454380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:16.454380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:16.454380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:16.454380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:16.454380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:16.454380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:16.454380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:16.454380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:16.454380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:16.454380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:16.454380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 164/337] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:49:16.458702 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e91ac-d1f2-4f78-a9d8-04a4145ee73a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:16.645161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:16.688042 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e91ac-d1f2-4f78-a9d8-04a4145ee73a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:16.738395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296141d-3d8d-4535-839d-aff33c7a5d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:16.739031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296141d-3d8d-4535-839d-aff33c7a5d21 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:16.739320 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296141d-3d8d-4535-839d-aff33c7a5d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:16.739602 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296141d-3d8d-4535-839d-aff33c7a5d21 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:16.740409 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296141d-3d8d-4535-839d-aff33c7a5d21 None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:16.740726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296141d-3d8d-4535-839d-aff33c7a5d21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:16.742154 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296141d-3d8d-4535-839d-aff33c7a5d21 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:16.756065 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 174/338] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:49:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:49:16.761969 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546e4d6-89e4-4c99-9185-789464bb8f96 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:16.766215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:16.918692 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546e4d6-89e4-4c99-9185-789464bb8f96 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:16.965093 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7cb3c-edd7-43d6-b29a-dfca0304270e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:16.965777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7cb3c-edd7-43d6-b29a-dfca0304270e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:16.966061 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7cb3c-edd7-43d6-b29a-dfca0304270e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:16.966333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7cb3c-edd7-43d6-b29a-dfca0304270e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:17.113749 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b7cb3c-edd7-43d6-b29a-dfca0304270e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:17.116881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b7cb3c-edd7-43d6-b29a-dfca0304270e None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:17.121219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:17.364517 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 165/339] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:49:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 16 14:49:17.374036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296141d-3d8d-4535-839d-aff33c7a5d21 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:17.625391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296141d-3d8d-4535-839d-aff33c7a5d21 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:17.691345 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:17.708587 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0187a3-aa68-463e-8c68-6b0e6c78fae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:17.710037 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0187a3-aa68-463e-8c68-6b0e6c78fae2 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:17.710537 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0187a3-aa68-463e-8c68-6b0e6c78fae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:17.711071 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0187a3-aa68-463e-8c68-6b0e6c78fae2 None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:17.717737 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0187a3-aa68-463e-8c68-6b0e6c78fae2 None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:17.720394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0187a3-aa68-463e-8c68-6b0e6c78fae2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:17.724088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 175/340] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:17 2022] GET /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532 => generated 298 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:17.733875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7cb3c-edd7-43d6-b29a-dfca0304270e None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:17.974222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7cb3c-edd7-43d6-b29a-dfca0304270e None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:17.977309 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:18.056184 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7d22c-ecbf-42bf-b7c5-fe567c13bbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:18.057697 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7d22c-ecbf-42bf-b7c5-fe567c13bbba None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:18.058199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7d22c-ecbf-42bf-b7c5-fe567c13bbba None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:18.058702 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7d22c-ecbf-42bf-b7c5-fe567c13bbba None admin] PATH_INFO: `/v3/groups/f99848359a454a4abf52ac354138df49` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:18.074175 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7d22c-ecbf-42bf-b7c5-fe567c13bbba None admin] RBAC: Authorizing `identity:get_group(group_id=f99848359a454a4abf52ac354138df49)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:18.076699 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7d22c-ecbf-42bf-b7c5-fe567c13bbba None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:18.080280 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 166/341] 10.222.0.27 () {56 vars in 1180 bytes} [Sat Jul 16 14:49:17 2022] GET /identity/v3/groups/f99848359a454a4abf52ac354138df49 => generated 226 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:49:18.090155 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0187a3-aa68-463e-8c68-6b0e6c78fae2 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:18.348274 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0187a3-aa68-463e-8c68-6b0e6c78fae2 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:18.420456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef4077-f650-4f20-bd80-b282ccae36c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:18.421629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef4077-f650-4f20-bd80-b282ccae36c0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:18.422086 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef4077-f650-4f20-bd80-b282ccae36c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:18.422572 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef4077-f650-4f20-bd80-b282ccae36c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:18.424257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef4077-f650-4f20-bd80-b282ccae36c0 None admin] RBAC: Adding query filter params (group.id=f99848359a454a4abf52ac354138df49, role.id=b7f92100b08f4dc6b2b6866a2955d2ab, scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:18.424798 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef4077-f650-4f20-bd80-b282ccae36c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:18.427597 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef4077-f650-4f20-bd80-b282ccae36c0 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:18.439634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 176/342] 10.222.0.27 () {56 vars in 1376 bytes} [Sat Jul 16 14:49:18 2022] GET /identity/v3/role_assignments?group.id=f99848359a454a4abf52ac354138df49&scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532&role.id=b7f92100b08f4dc6b2b6866a2955d2ab => generated 260 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:18.857252 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c8d4e-0a9b-42c1-8dae-115782bd0861 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:18.858283 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c8d4e-0a9b-42c1-8dae-115782bd0861 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:18.858283 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c8d4e-0a9b-42c1-8dae-115782bd0861 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:18.861264 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 167/343] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:18.870049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e236f20-e266-4638-a075-3359376489c2 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:18.870426 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e236f20-e266-4638-a075-3359376489c2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:18.870654 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e236f20-e266-4638-a075-3359376489c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:18.914338 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-0e236f20-e266-4638-a075-3359376489c2 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:18.917526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-0e236f20-e266-4638-a075-3359376489c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:19.072277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:19.098390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 177/344] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:19.138955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c8d4e-0a9b-42c1-8dae-115782bd0861 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:19.155176 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbe60c-7aad-4fb2-84ec-40cef84212dd None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:19.155948 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbe60c-7aad-4fb2-84ec-40cef84212dd None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:19.156543 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbe60c-7aad-4fb2-84ec-40cef84212dd None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:19.160605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 178/345] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:19.172055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08ba86-bf64-47ca-b543-baa9ea4dd2b7 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:19.172833 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08ba86-bf64-47ca-b543-baa9ea4dd2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:19.173479 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08ba86-bf64-47ca-b543-baa9ea4dd2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:19.222879 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ea08ba86-bf64-47ca-b543-baa9ea4dd2b7 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:19.225001 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ea08ba86-bf64-47ca-b543-baa9ea4dd2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:19.400775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 179/346] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:19.403260 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d3d91-17ae-4137-9fe0-faf4f033fd28 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:19.403759 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d3d91-17ae-4137-9fe0-faf4f033fd28 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:19.404244 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d3d91-17ae-4137-9fe0-faf4f033fd28 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:19.407141 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 180/347] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:19.407984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c8d4e-0a9b-42c1-8dae-115782bd0861 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:19.409047 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92b1d7-9aab-42fa-a269-d9be3401da3b None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:19.409553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92b1d7-9aab-42fa-a269-d9be3401da3b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:19.410038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92b1d7-9aab-42fa-a269-d9be3401da3b None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:19.412953 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 181/348] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:19.425344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5e585-b892-44de-bd64-9dc84827a537 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:19.426280 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5e585-b892-44de-bd64-9dc84827a537 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:19.426280 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5e585-b892-44de-bd64-9dc84827a537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:19.454100 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-05d5e585-b892-44de-bd64-9dc84827a537 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:19.455370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-05d5e585-b892-44de-bd64-9dc84827a537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:19.481664 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d354bd-4082-42d6-a6c8-1371438c72e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:19.483591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d354bd-4082-42d6-a6c8-1371438c72e7 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:19.484337 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d354bd-4082-42d6-a6c8-1371438c72e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:19.485035 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d354bd-4082-42d6-a6c8-1371438c72e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:19.516625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d354bd-4082-42d6-a6c8-1371438c72e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:19.533839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-d6d354bd-4082-42d6-a6c8-1371438c72e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:49:19.533839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:19.533839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:19.533839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:19.533839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:19.533839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:19.533839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:19.533839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:19.533839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:19.533839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:19.533839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:19.533839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:19.533839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:19.533839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:19.533839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:19.533839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:19.535672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:19.535672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:19.535672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:19.535672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:19.535672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:19.535672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:19.535672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:19.535672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:19.535672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:19.535672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:19.535672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:19.535672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:19.535672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:19.535672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:19.535672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:19.535672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:19.537363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:19.537363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:19.537363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:19.537363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:19.537363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:19.537363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:19.537363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:19.537363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:19.537363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:19.537363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:19.537363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:49:19.537363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:19.541245 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 168/349] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:49:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:19.543916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad849841-ef08-4a11-8739-e2ddafbd93cd None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:19.544719 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad849841-ef08-4a11-8739-e2ddafbd93cd None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:19.545451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad849841-ef08-4a11-8739-e2ddafbd93cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:19.568128 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 182/350] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 14:49:19.569156 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5e585-b892-44de-bd64-9dc84827a537 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:19.575882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-ad849841-ef08-4a11-8739-e2ddafbd93cd None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:19.578316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ad849841-ef08-4a11-8739-e2ddafbd93cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:19.625452 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:19.691691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:19.697477 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 169/351] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:19.698123 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad849841-ef08-4a11-8739-e2ddafbd93cd None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:19.756804 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5e585-b892-44de-bd64-9dc84827a537 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:19.843123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f095633-9781-4cfc-b6d5-ebb056f9cd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:19.844191 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f095633-9781-4cfc-b6d5-ebb056f9cd16 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:19.844618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f095633-9781-4cfc-b6d5-ebb056f9cd16 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:19.845064 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f095633-9781-4cfc-b6d5-ebb056f9cd16 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:19.869908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f095633-9781-4cfc-b6d5-ebb056f9cd16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:19.885232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad849841-ef08-4a11-8739-e2ddafbd93cd None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:19.891884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-8f095633-9781-4cfc-b6d5-ebb056f9cd16 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:19.891884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:19.891884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:19.891884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:19.891884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:19.891884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:19.891884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:19.891884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:19.891884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:19.891884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:19.891884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:19.891884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:19.891884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:19.891884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:19.891884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:19.891884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:19.892908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:19.892908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:19.892908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:19.892908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:19.892908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:19.892908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:19.892908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:19.892908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:19.892908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:19.892908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:19.892908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:19.892908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:19.892908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:19.892908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:19.892908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:19.892908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:19.893719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:19.893719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:19.893719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:19.893719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:19.893719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:19.893719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:19.893719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:19.893719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:19.893719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:19.893719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:19.893719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:19.893719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:19.894775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 183/352] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:19.896349 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f095633-9781-4cfc-b6d5-ebb056f9cd16 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:19.942695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d8277-16ac-477b-839c-bb40a5641b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:19.943785 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d8277-16ac-477b-839c-bb40a5641b14 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:19.944267 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d8277-16ac-477b-839c-bb40a5641b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:19.944923 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d8277-16ac-477b-839c-bb40a5641b14 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:19.947312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d8277-16ac-477b-839c-bb40a5641b14 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:19.947878 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d8277-16ac-477b-839c-bb40a5641b14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:19.949518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d8277-16ac-477b-839c-bb40a5641b14 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:19.962536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 170/353] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:49:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:19.964303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d8277-16ac-477b-839c-bb40a5641b14 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:19.994540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:20.142014 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d8277-16ac-477b-839c-bb40a5641b14 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:20.155681 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f095633-9781-4cfc-b6d5-ebb056f9cd16 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:20.181055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:20.212899 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffde98b-0aca-49cf-930b-2e9060819398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:20.214133 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffde98b-0aca-49cf-930b-2e9060819398 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:20.214763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffde98b-0aca-49cf-930b-2e9060819398 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:20.215278 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffde98b-0aca-49cf-930b-2e9060819398 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:20.224349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ba474-62cc-4abb-92b1-b8389265209e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:20.226033 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ba474-62cc-4abb-92b1-b8389265209e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:20.226541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ba474-62cc-4abb-92b1-b8389265209e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:20.227591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ba474-62cc-4abb-92b1-b8389265209e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:20.235690 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffde98b-0aca-49cf-930b-2e9060819398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:20.248159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ba474-62cc-4abb-92b1-b8389265209e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:20.252340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-9ffde98b-0aca-49cf-930b-2e9060819398 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 14:49:20.252340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:20.252340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:20.252340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:20.252340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:20.252340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:20.252340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:20.252340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:20.252340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:20.252340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:20.252340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:20.252340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:20.252340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:20.252340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:20.252340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:20.252340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:20.253243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:20.253243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:20.253243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:20.253243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:20.253243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:20.253243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:20.253243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:20.253243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:20.253243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:20.253243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:20.253243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:20.253243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:20.253243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:20.253243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:20.253243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:20.253243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:20.254017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:20.254017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:20.254017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:20.254017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:20.254017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:20.254017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:20.254017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:20.254017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:20.254017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:20.254017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:20.254017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 16 14:49:20.254017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:20.256531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 184/354] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:20.257694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffde98b-0aca-49cf-930b-2e9060819398 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:20.264807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-168ba474-62cc-4abb-92b1-b8389265209e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:49:20.264807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:20.264807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:20.264807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:20.264807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:20.264807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:20.264807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:20.264807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:20.264807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:20.264807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:20.264807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:20.264807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:20.264807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:20.264807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:20.264807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:20.264807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:20.265782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:20.265782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:20.265782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:20.265782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:20.265782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:20.265782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:20.265782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:20.265782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:20.265782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:20.265782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:20.265782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:20.265782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:20.265782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:20.265782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:20.265782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:20.265782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:20.266600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:20.266600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:20.266600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:20.266600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:20.266600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:20.266600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:20.266600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:20.266600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:20.266600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:20.266600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:20.266600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:49:20.266600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:20.267333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 171/355] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:49:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:20.268584 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ba474-62cc-4abb-92b1-b8389265209e None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:20.412216 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffde98b-0aca-49cf-930b-2e9060819398 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:20.422197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:20.454154 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192afea6-8308-49ad-be8e-738ef2b9dca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:20.455602 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192afea6-8308-49ad-be8e-738ef2b9dca5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:20.456119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192afea6-8308-49ad-be8e-738ef2b9dca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:20.456617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192afea6-8308-49ad-be8e-738ef2b9dca5 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:20.457587 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192afea6-8308-49ad-be8e-738ef2b9dca5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:20.458098 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192afea6-8308-49ad-be8e-738ef2b9dca5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:20.459993 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192afea6-8308-49ad-be8e-738ef2b9dca5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:20.466441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ba474-62cc-4abb-92b1-b8389265209e None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:20.471864 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 185/356] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:49:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:20.473078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192afea6-8308-49ad-be8e-738ef2b9dca5 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:20.514378 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:20.541628 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93259bc0-2fb7-4974-8d3a-4ca044cf61d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:20.542033 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93259bc0-2fb7-4974-8d3a-4ca044cf61d2 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:20.542545 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93259bc0-2fb7-4974-8d3a-4ca044cf61d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:20.543048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93259bc0-2fb7-4974-8d3a-4ca044cf61d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:20.621483 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192afea6-8308-49ad-be8e-738ef2b9dca5 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:20.674628 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93259bc0-2fb7-4974-8d3a-4ca044cf61d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:20.676597 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93259bc0-2fb7-4974-8d3a-4ca044cf61d2 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:20.694778 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e518c8-3c60-4786-b701-fbcb8de78b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:20.695596 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e518c8-3c60-4786-b701-fbcb8de78b91 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:20.696192 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e518c8-3c60-4786-b701-fbcb8de78b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:20.696644 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e518c8-3c60-4786-b701-fbcb8de78b91 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:20.697499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e518c8-3c60-4786-b701-fbcb8de78b91 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:20.697875 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e518c8-3c60-4786-b701-fbcb8de78b91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:20.699411 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e518c8-3c60-4786-b701-fbcb8de78b91 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:20.721804 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 186/357] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:49:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:20.721804 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e518c8-3c60-4786-b701-fbcb8de78b91 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:20.832106 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:20.869938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 172/358] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:49:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:49:20.871760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93259bc0-2fb7-4974-8d3a-4ca044cf61d2 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:20.880487 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e518c8-3c60-4786-b701-fbcb8de78b91 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:20.884003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:20.943671 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa3ac9-e790-4dc6-bbe7-b18f71713a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:20.945164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa3ac9-e790-4dc6-bbe7-b18f71713a19 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:20.945968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa3ac9-e790-4dc6-bbe7-b18f71713a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:20.946701 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa3ac9-e790-4dc6-bbe7-b18f71713a19 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:20.948291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa3ac9-e790-4dc6-bbe7-b18f71713a19 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:20.949085 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa3ac9-e790-4dc6-bbe7-b18f71713a19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:20.951618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa3ac9-e790-4dc6-bbe7-b18f71713a19 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:20.971728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 187/359] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:49:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:20.973327 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da4e50-c38f-47c1-b3cd-516d6e66add6 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:20.973616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da4e50-c38f-47c1-b3cd-516d6e66add6 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:20.973898 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da4e50-c38f-47c1-b3cd-516d6e66add6 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:20.975663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 188/360] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:20.976130 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da4e50-c38f-47c1-b3cd-516d6e66add6 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:21.083721 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93259bc0-2fb7-4974-8d3a-4ca044cf61d2 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:21.142646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da4e50-c38f-47c1-b3cd-516d6e66add6 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:21.166730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102cde8a-b1de-4edc-b050-897c329ff7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:21.167584 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cde8a-b1de-4edc-b050-897c329ff7fb None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:21.167740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cde8a-b1de-4edc-b050-897c329ff7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:21.167987 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cde8a-b1de-4edc-b050-897c329ff7fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:21.213746 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd3885-4b71-4c48-8ade-8b6981cf9f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:21.216033 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd3885-4b71-4c48-8ade-8b6981cf9f3c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:21.216524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd3885-4b71-4c48-8ade-8b6981cf9f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:21.217180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd3885-4b71-4c48-8ade-8b6981cf9f3c None admin] PATH_INFO: `/v3/users/e3b7400e8bf44a7f8874c897cb2e1980` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:21.222998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd3885-4b71-4c48-8ade-8b6981cf9f3c None admin] RBAC: Authorizing `identity:get_user(user_id=e3b7400e8bf44a7f8874c897cb2e1980)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:21.225733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd3885-4b71-4c48-8ade-8b6981cf9f3c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:21.230183 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 189/361] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:49:20 2022] GET /identity/v3/users/e3b7400e8bf44a7f8874c897cb2e1980 => generated 293 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:21.232265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd3885-4b71-4c48-8ade-8b6981cf9f3c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:21.237644 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:21.288674 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:21.296730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102cde8a-b1de-4edc-b050-897c329ff7fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:21.299374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102cde8a-b1de-4edc-b050-897c329ff7fb None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:21.407388 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd3885-4b71-4c48-8ade-8b6981cf9f3c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:21.465912 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c846c-304f-4443-9191-d65379124134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:21.466918 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c846c-304f-4443-9191-d65379124134 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:21.467226 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c846c-304f-4443-9191-d65379124134 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:21.467502 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c846c-304f-4443-9191-d65379124134 None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:21.468814 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c846c-304f-4443-9191-d65379124134 None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:21.470340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c846c-304f-4443-9191-d65379124134 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:21.472564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 190/362] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:21 2022] GET /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:21.473485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c846c-304f-4443-9191-d65379124134 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:21.503983 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 173/363] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:49:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:49:21.506042 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b817ec0-c01f-44dc-a5c1-59167c588752 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:21.506405 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b817ec0-c01f-44dc-a5c1-59167c588752 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:21.506968 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b817ec0-c01f-44dc-a5c1-59167c588752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:21.536381 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-1b817ec0-c01f-44dc-a5c1-59167c588752 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:21.538532 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-1b817ec0-c01f-44dc-a5c1-59167c588752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:21.656647 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:21.672965 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c846c-304f-4443-9191-d65379124134 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:21.682147 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 174/364] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:21.683678 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b817ec0-c01f-44dc-a5c1-59167c588752 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:21.707472 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:21.746701 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1f1fe-b172-4c8a-9167-333d01308531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:21.747817 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1f1fe-b172-4c8a-9167-333d01308531 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:21.748288 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1f1fe-b172-4c8a-9167-333d01308531 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:21.748769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1f1fe-b172-4c8a-9167-333d01308531 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:21.908963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b817ec0-c01f-44dc-a5c1-59167c588752 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:21.912584 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1f1fe-b172-4c8a-9167-333d01308531 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:21.916663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1f1fe-b172-4c8a-9167-333d01308531 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:21.986496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9636651-d5f8-46e8-9503-73a66c7ec274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:21.987522 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9636651-d5f8-46e8-9503-73a66c7ec274 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:21.988491 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9636651-d5f8-46e8-9503-73a66c7ec274 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:21.988971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9636651-d5f8-46e8-9503-73a66c7ec274 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:21.990591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9636651-d5f8-46e8-9503-73a66c7ec274 None admin] RBAC: Adding query filter params (role.id=7f966f669d82448daddfe911e2955ea2, user.id=e3b7400e8bf44a7f8874c897cb2e1980, scope.system=all) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:21.991340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9636651-d5f8-46e8-9503-73a66c7ec274 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:21.994207 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9636651-d5f8-46e8-9503-73a66c7ec274 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:22.015867 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 175/365] 10.222.0.27 () {56 vars in 1308 bytes} [Sat Jul 16 14:49:21 2022] GET /identity/v3/role_assignments?user.id=e3b7400e8bf44a7f8874c897cb2e1980&scope.system=all&role.id=7f966f669d82448daddfe911e2955ea2 => generated 510 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:22.020373 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9636651-d5f8-46e8-9503-73a66c7ec274 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:22.096451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:22.109923 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 191/366] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:49:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:49:22.111027 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1f1fe-b172-4c8a-9167-333d01308531 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:22.138183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:22.205719 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9636651-d5f8-46e8-9503-73a66c7ec274 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:22.276223 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1825d0-aa6f-4c0c-b0bb-3ca02cb7faeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:22.277973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1825d0-aa6f-4c0c-b0bb-3ca02cb7faeb None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:22.278629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1825d0-aa6f-4c0c-b0bb-3ca02cb7faeb None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:22.279931 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1825d0-aa6f-4c0c-b0bb-3ca02cb7faeb None admin] PATH_INFO: `/v3/groups/f99848359a454a4abf52ac354138df49` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:22.281817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1825d0-aa6f-4c0c-b0bb-3ca02cb7faeb None admin] RBAC: Authorizing `identity:get_group(group_id=f99848359a454a4abf52ac354138df49)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:22.284489 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1825d0-aa6f-4c0c-b0bb-3ca02cb7faeb None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:22.287325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 176/367] 10.222.0.27 () {56 vars in 1180 bytes} [Sat Jul 16 14:49:22 2022] GET /identity/v3/groups/f99848359a454a4abf52ac354138df49 => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:22.294983 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1825d0-aa6f-4c0c-b0bb-3ca02cb7faeb None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:22.318142 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1f1fe-b172-4c8a-9167-333d01308531 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:22.377268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc061ff-126b-496b-ac70-1df11833b294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:22.378223 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc061ff-126b-496b-ac70-1df11833b294 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:22.378560 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc061ff-126b-496b-ac70-1df11833b294 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:22.378952 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc061ff-126b-496b-ac70-1df11833b294 None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:22.380978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc061ff-126b-496b-ac70-1df11833b294 None admin] RBAC: Authorizing `identity:get_project(project_id=aad3af41314e48b4a48d335366f333ce)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:22.383611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc061ff-126b-496b-ac70-1df11833b294 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:22.385593 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 192/368] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:22 2022] GET /identity/v3/projects/aad3af41314e48b4a48d335366f333ce => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:22.387359 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc239ec0-1a00-4dd5-91ba-40f1abe45833 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:22.387829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc239ec0-1a00-4dd5-91ba-40f1abe45833 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:22.388291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc239ec0-1a00-4dd5-91ba-40f1abe45833 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:22.390722 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 193/369] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:22.391898 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc239ec0-1a00-4dd5-91ba-40f1abe45833 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:22.485534 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1825d0-aa6f-4c0c-b0bb-3ca02cb7faeb None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:22.508879 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:22.551255 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91539a-c9fd-47cc-b9b0-d2041cbb593e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:22.552071 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91539a-c9fd-47cc-b9b0-d2041cbb593e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:22.552355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91539a-c9fd-47cc-b9b0-d2041cbb593e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:22.552628 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91539a-c9fd-47cc-b9b0-d2041cbb593e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:22.554919 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:22.571672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91539a-c9fd-47cc-b9b0-d2041cbb593e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:22.572082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc239ec0-1a00-4dd5-91ba-40f1abe45833 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:22.585044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-2e91539a-c9fd-47cc-b9b0-d2041cbb593e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:49:22.585044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:22.585044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:22.585044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:22.585044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:22.585044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:22.585044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:22.585044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:22.585044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:22.585044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:22.585044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:22.585044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:22.585044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:22.585044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:22.585044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:22.585044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:22.586586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:22.586586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:22.586586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:22.586586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:22.586586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:22.586586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:22.586586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:22.586586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:22.586586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:22.586586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:22.586586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:22.586586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:22.586586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:22.586586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:22.586586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:22.586586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:22.588367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:22.588367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:22.588367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:22.588367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:22.588367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:22.588367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:22.588367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:22.588367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:22.588367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:22.588367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:22.588367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:49:22.588367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:22.591582 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 177/370] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:49:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:22.592616 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91539a-c9fd-47cc-b9b0-d2041cbb593e None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:22.611586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf53b6-b814-4d53-b3a6-a6dca5fe2b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:22.612455 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf53b6-b814-4d53-b3a6-a6dca5fe2b6e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:22.612730 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf53b6-b814-4d53-b3a6-a6dca5fe2b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:22.612999 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf53b6-b814-4d53-b3a6-a6dca5fe2b6e None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:22.614213 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf53b6-b814-4d53-b3a6-a6dca5fe2b6e None admin] RBAC: Authorizing `identity:get_project(project_id=aad3af41314e48b4a48d335366f333ce)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:22.615679 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf53b6-b814-4d53-b3a6-a6dca5fe2b6e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:22.617420 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 194/371] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:22 2022] GET /identity/v3/projects/aad3af41314e48b4a48d335366f333ce => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:22.618763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf53b6-b814-4d53-b3a6-a6dca5fe2b6e None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:22.732780 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91539a-c9fd-47cc-b9b0-d2041cbb593e None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:22.739586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf53b6-b814-4d53-b3a6-a6dca5fe2b6e None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:22.775349 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:22.777646 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e67117-5249-4a5d-881a-f07b22d7b8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:22.778313 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e67117-5249-4a5d-881a-f07b22d7b8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:22.778591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e67117-5249-4a5d-881a-f07b22d7b8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:22.778858 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e67117-5249-4a5d-881a-f07b22d7b8b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:22.779741 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e67117-5249-4a5d-881a-f07b22d7b8b9 None admin] RBAC: Adding query filter params (group.id=f99848359a454a4abf52ac354138df49, role.id=af11cd5265d14ac7902b1fb3afcc4be7, scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:22.780049 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e67117-5249-4a5d-881a-f07b22d7b8b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:22.781636 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e67117-5249-4a5d-881a-f07b22d7b8b9 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:22.782339 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe977b-5a2d-44b2-a217-4028c08fa32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:22.783337 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe977b-5a2d-44b2-a217-4028c08fa32a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:22.783794 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe977b-5a2d-44b2-a217-4028c08fa32a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:22.785633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe977b-5a2d-44b2-a217-4028c08fa32a None admin] PATH_INFO: `/v3/groups/f99848359a454a4abf52ac354138df49` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:22.785809 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe977b-5a2d-44b2-a217-4028c08fa32a None admin] RBAC: Authorizing `identity:get_group(group_id=f99848359a454a4abf52ac354138df49)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:22.787606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe977b-5a2d-44b2-a217-4028c08fa32a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:22.788126 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 178/372] 10.222.0.27 () {56 vars in 1376 bytes} [Sat Jul 16 14:49:22 2022] GET /identity/v3/role_assignments?group.id=f99848359a454a4abf52ac354138df49&scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532&role.id=af11cd5265d14ac7902b1fb3afcc4be7 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:22.790084 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96cb69-a18c-49fb-a093-4dcd98cb06f6 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:22.796483 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 195/373] 10.222.0.27 () {56 vars in 1180 bytes} [Sat Jul 16 14:49:22 2022] GET /identity/v3/groups/f99848359a454a4abf52ac354138df49 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:22.796685 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96cb69-a18c-49fb-a093-4dcd98cb06f6 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:22.796685 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd96cb69-a18c-49fb-a093-4dcd98cb06f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:22.796838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe977b-5a2d-44b2-a217-4028c08fa32a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:22.822023 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-cd96cb69-a18c-49fb-a093-4dcd98cb06f6 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:22.824167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-cd96cb69-a18c-49fb-a093-4dcd98cb06f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:22.926709 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:22.950950 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 179/374] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:22.950950 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96cb69-a18c-49fb-a093-4dcd98cb06f6 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:22.999028 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe977b-5a2d-44b2-a217-4028c08fa32a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:23.058673 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82346aa-7921-4a84-a179-ec3b66b96294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:23.059981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82346aa-7921-4a84-a179-ec3b66b96294 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:23.060480 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82346aa-7921-4a84-a179-ec3b66b96294 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:23.060980 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82346aa-7921-4a84-a179-ec3b66b96294 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:23.062424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82346aa-7921-4a84-a179-ec3b66b96294 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:23.062772 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82346aa-7921-4a84-a179-ec3b66b96294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:23.065127 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82346aa-7921-4a84-a179-ec3b66b96294 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:23.080407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 196/375] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:23.081857 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd96cb69-a18c-49fb-a093-4dcd98cb06f6 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:23.081971 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82346aa-7921-4a84-a179-ec3b66b96294 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:23.133518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33dc893-0898-42b9-bd76-891c1d4d7a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:23.134148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33dc893-0898-42b9-bd76-891c1d4d7a93 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:23.134577 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33dc893-0898-42b9-bd76-891c1d4d7a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:23.134850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33dc893-0898-42b9-bd76-891c1d4d7a93 None admin] PATH_INFO: `/v3/groups/f99848359a454a4abf52ac354138df49` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:23.136155 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33dc893-0898-42b9-bd76-891c1d4d7a93 None admin] RBAC: Authorizing `identity:get_group(group_id=f99848359a454a4abf52ac354138df49)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:23.137630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33dc893-0898-42b9-bd76-891c1d4d7a93 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:23.143540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 180/376] 10.222.0.27 () {56 vars in 1180 bytes} [Sat Jul 16 14:49:22 2022] GET /identity/v3/groups/f99848359a454a4abf52ac354138df49 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:23.145304 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33dc893-0898-42b9-bd76-891c1d4d7a93 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:23.167199 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:23.224317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:23.224516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82346aa-7921-4a84-a179-ec3b66b96294 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:23.278230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33dc893-0898-42b9-bd76-891c1d4d7a93 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:23.278468 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e20f2-d2b8-4c74-ba75-eca93d12028b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:23.279132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e20f2-d2b8-4c74-ba75-eca93d12028b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:23.279412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e20f2-d2b8-4c74-ba75-eca93d12028b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:23.279789 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e20f2-d2b8-4c74-ba75-eca93d12028b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:23.280697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e20f2-d2b8-4c74-ba75-eca93d12028b None admin] RBAC: Adding query filter params (group.id=f99848359a454a4abf52ac354138df49, role.id=b7f92100b08f4dc6b2b6866a2955d2ab, scope.project.id=aad3af41314e48b4a48d335366f333ce) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:23.281038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e20f2-d2b8-4c74-ba75-eca93d12028b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:23.282702 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e20f2-d2b8-4c74-ba75-eca93d12028b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:23.299365 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 197/377] 10.222.0.27 () {56 vars in 1376 bytes} [Sat Jul 16 14:49:23 2022] GET /identity/v3/role_assignments?group.id=f99848359a454a4abf52ac354138df49&scope.project.id=aad3af41314e48b4a48d335366f333ce&role.id=b7f92100b08f4dc6b2b6866a2955d2ab => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:23.299365 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e20f2-d2b8-4c74-ba75-eca93d12028b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:23.343377 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4748677-c9a0-40f5-9ef2-4d2491f6a01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:23.344447 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4748677-c9a0-40f5-9ef2-4d2491f6a01e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:23.344824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4748677-c9a0-40f5-9ef2-4d2491f6a01e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:23.345739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4748677-c9a0-40f5-9ef2-4d2491f6a01e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:23.364811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4748677-c9a0-40f5-9ef2-4d2491f6a01e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:23.377205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-e4748677-c9a0-40f5-9ef2-4d2491f6a01e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:23.377205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:23.377205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:23.377205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:23.377205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:23.377205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:23.377205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:23.377205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:23.377205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:23.377205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:23.377205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:23.377205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:23.377205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:23.377205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:23.377205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:23.377205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:23.378863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:23.378863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:23.378863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:23.378863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:23.378863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:23.378863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:23.378863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:23.378863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:23.378863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:23.378863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:23.378863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:23.378863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:23.378863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:23.378863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:23.378863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:23.378863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:23.380454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:23.380454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:23.380454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:23.380454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:23.380454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:23.380454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:23.380454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:23.380454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:23.380454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:23.380454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:23.380454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:23.380454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:23.380454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 181/378] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:23.383785 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4748677-c9a0-40f5-9ef2-4d2491f6a01e None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:23.428614 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e20f2-d2b8-4c74-ba75-eca93d12028b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:23.472013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:23.487267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c5543-d4fa-4d98-bbab-eb2383b434e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:23.488034 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c5543-d4fa-4d98-bbab-eb2383b434e5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:23.488317 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c5543-d4fa-4d98-bbab-eb2383b434e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:23.488592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c5543-d4fa-4d98-bbab-eb2383b434e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:23.567362 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4748677-c9a0-40f5-9ef2-4d2491f6a01e None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:23.601241 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:23.625434 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c5543-d4fa-4d98-bbab-eb2383b434e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:23.628815 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c5543-d4fa-4d98-bbab-eb2383b434e5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:23.629542 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933489d-5440-4d3e-8643-50d4f8e0ad17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:23.630877 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933489d-5440-4d3e-8643-50d4f8e0ad17 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:23.631573 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933489d-5440-4d3e-8643-50d4f8e0ad17 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:23.632223 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933489d-5440-4d3e-8643-50d4f8e0ad17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:23.635256 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933489d-5440-4d3e-8643-50d4f8e0ad17 None admin] RBAC: Adding query filter params (group.id=f99848359a454a4abf52ac354138df49, role.id=af11cd5265d14ac7902b1fb3afcc4be7, scope.project.id=aad3af41314e48b4a48d335366f333ce) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:23.636325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933489d-5440-4d3e-8643-50d4f8e0ad17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:23.641085 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933489d-5440-4d3e-8643-50d4f8e0ad17 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:23.659590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 182/379] 10.222.0.27 () {56 vars in 1376 bytes} [Sat Jul 16 14:49:23 2022] GET /identity/v3/role_assignments?group.id=f99848359a454a4abf52ac354138df49&scope.project.id=aad3af41314e48b4a48d335366f333ce&role.id=af11cd5265d14ac7902b1fb3afcc4be7 => generated 260 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:23.659590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933489d-5440-4d3e-8643-50d4f8e0ad17 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:23.979082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 198/380] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:49:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 684 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:49:23.983491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c5543-d4fa-4d98-bbab-eb2383b434e5 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:23.993051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933489d-5440-4d3e-8643-50d4f8e0ad17 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:24.036754 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:24.064150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85ee2a-04ba-431f-8302-49fa29fd0629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:24.065722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85ee2a-04ba-431f-8302-49fa29fd0629 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:24.066454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85ee2a-04ba-431f-8302-49fa29fd0629 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:24.067237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85ee2a-04ba-431f-8302-49fa29fd0629 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:24.068895 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85ee2a-04ba-431f-8302-49fa29fd0629 None admin] RBAC: Adding query filter params (name=member) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:24.069772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85ee2a-04ba-431f-8302-49fa29fd0629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:24.073399 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca85ee2a-04ba-431f-8302-49fa29fd0629 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:24.091562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 183/381] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:49:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:24.100740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85ee2a-04ba-431f-8302-49fa29fd0629 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:24.188782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:24.190495 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c5543-d4fa-4d98-bbab-eb2383b434e5 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:24.310672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0600a8-3d3c-466f-ac59-cab2f89157a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:24.312074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0600a8-3d3c-466f-ac59-cab2f89157a6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:24.312553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0600a8-3d3c-466f-ac59-cab2f89157a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:24.313043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0600a8-3d3c-466f-ac59-cab2f89157a6 None admin] PATH_INFO: `/v3/projects/094ab8f948f540f7a205e832cbae4778` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:24.319404 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0600a8-3d3c-466f-ac59-cab2f89157a6 None admin] RBAC: Authorizing `identity:get_project(project_id=094ab8f948f540f7a205e832cbae4778)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:24.323798 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0600a8-3d3c-466f-ac59-cab2f89157a6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:24.326148 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 199/382] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:23 2022] GET /identity/v3/projects/094ab8f948f540f7a205e832cbae4778 => generated 344 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:49:24.339460 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0600a8-3d3c-466f-ac59-cab2f89157a6 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:24.409793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85ee2a-04ba-431f-8302-49fa29fd0629 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:24.510421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ee517-7b7c-4057-a7ad-0b870fb6cefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:24.511384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ee517-7b7c-4057-a7ad-0b870fb6cefe None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:24.511728 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ee517-7b7c-4057-a7ad-0b870fb6cefe None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:24.511977 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ee517-7b7c-4057-a7ad-0b870fb6cefe None admin] PATH_INFO: `/v3/groups/f99848359a454a4abf52ac354138df49` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:24.513430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ee517-7b7c-4057-a7ad-0b870fb6cefe None admin] RBAC: Authorizing `identity:get_group(group_id=f99848359a454a4abf52ac354138df49)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:24.515088 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ee517-7b7c-4057-a7ad-0b870fb6cefe None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:24.523438 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 184/383] 10.222.0.27 () {56 vars in 1180 bytes} [Sat Jul 16 14:49:24 2022] GET /identity/v3/groups/f99848359a454a4abf52ac354138df49 => generated 226 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:24.531953 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ee517-7b7c-4057-a7ad-0b870fb6cefe None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:24.579629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:24.595895 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0600a8-3d3c-466f-ac59-cab2f89157a6 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:24.663785 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66504a04-5963-4e0c-866b-ed51feda9164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:24.665817 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66504a04-5963-4e0c-866b-ed51feda9164 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:24.666377 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66504a04-5963-4e0c-866b-ed51feda9164 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:24.666783 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66504a04-5963-4e0c-866b-ed51feda9164 None admin] PATH_INFO: `/v3/groups/dcdb8477e03e41fda3c3739065ecc097` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:24.678670 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66504a04-5963-4e0c-866b-ed51feda9164 None admin] RBAC: Authorizing `identity:get_group(group_id=dcdb8477e03e41fda3c3739065ecc097)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:24.682067 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66504a04-5963-4e0c-866b-ed51feda9164 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:24.690161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 200/384] 10.222.0.27 () {56 vars in 1180 bytes} [Sat Jul 16 14:49:24 2022] GET /identity/v3/groups/dcdb8477e03e41fda3c3739065ecc097 => generated 229 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:24.695939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66504a04-5963-4e0c-866b-ed51feda9164 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:24.914515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66504a04-5963-4e0c-866b-ed51feda9164 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:24.930569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ee517-7b7c-4057-a7ad-0b870fb6cefe None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:24.974072 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:24.994302 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0aba7-b4ef-4cb8-acc2-8389233e06d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:24.995978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0aba7-b4ef-4cb8-acc2-8389233e06d8 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:24.997063 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0aba7-b4ef-4cb8-acc2-8389233e06d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:24.997898 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0aba7-b4ef-4cb8-acc2-8389233e06d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:24.999893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0aba7-b4ef-4cb8-acc2-8389233e06d8 None admin] RBAC: Adding query filter params (group.id=dcdb8477e03e41fda3c3739065ecc097, role.id=0efeff46573247f3b9f1e2dec2968672, scope.project.id=094ab8f948f540f7a205e832cbae4778) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:25.001400 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0aba7-b4ef-4cb8-acc2-8389233e06d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:25.004618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0aba7-b4ef-4cb8-acc2-8389233e06d8 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:25.018714 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 201/385] 10.222.0.27 () {56 vars in 1376 bytes} [Sat Jul 16 14:49:24 2022] GET /identity/v3/role_assignments?group.id=dcdb8477e03e41fda3c3739065ecc097&scope.project.id=094ab8f948f540f7a205e832cbae4778&role.id=0efeff46573247f3b9f1e2dec2968672 => generated 260 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:49:25.056337 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b85a559-da9d-44ea-9262-290074950ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:25.057950 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85a559-da9d-44ea-9262-290074950ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:25.058448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85a559-da9d-44ea-9262-290074950ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:25.059121 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85a559-da9d-44ea-9262-290074950ba1 None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:25.061488 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85a559-da9d-44ea-9262-290074950ba1 None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:25.064435 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b85a559-da9d-44ea-9262-290074950ba1 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:25.074037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 185/386] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:24 2022] GET /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532 => generated 298 bytes in 537 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:49:25.077673 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0aba7-b4ef-4cb8-acc2-8389233e06d8 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:25.129533 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:25.316426 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0aba7-b4ef-4cb8-acc2-8389233e06d8 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:25.400829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e694fe-c88c-4892-aed1-7ee6b495de75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:25.402848 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e694fe-c88c-4892-aed1-7ee6b495de75 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:25.403491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e694fe-c88c-4892-aed1-7ee6b495de75 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:25.403993 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e694fe-c88c-4892-aed1-7ee6b495de75 None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532/groups/f99848359a454a4abf52ac354138df49/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:25.413572 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e694fe-c88c-4892-aed1-7ee6b495de75 None admin] RBAC: Authorizing `identity:create_grant(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532, group_id=f99848359a454a4abf52ac354138df49, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:25.417768 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e694fe-c88c-4892-aed1-7ee6b495de75 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:25.519377 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 202/387] 10.222.0.27 () {58 vars in 1442 bytes} [Sat Jul 16 14:49:25 2022] PUT /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532/groups/f99848359a454a4abf52ac354138df49/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:49:26.029587 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c347c9-b403-429b-a5ac-194555f3d538 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:26.030091 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c347c9-b403-429b-a5ac-194555f3d538 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:26.030570 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c347c9-b403-429b-a5ac-194555f3d538 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:26.033723 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 186/388] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:26.041218 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e626a-a73c-4dd3-ba99-15f784637812 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:26.041721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e626a-a73c-4dd3-ba99-15f784637812 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:26.042164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e626a-a73c-4dd3-ba99-15f784637812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:26.088616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-638e626a-a73c-4dd3-ba99-15f784637812 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:26.090677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-638e626a-a73c-4dd3-ba99-15f784637812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:26.231922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 203/389] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:26.271396 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c347c9-b403-429b-a5ac-194555f3d538 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:26.516265 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c347c9-b403-429b-a5ac-194555f3d538 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:26.527875 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27354388-88fe-4054-a3d3-7b046bfc42a9 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:26.528393 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27354388-88fe-4054-a3d3-7b046bfc42a9 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:26.528899 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27354388-88fe-4054-a3d3-7b046bfc42a9 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:26.541010 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 204/390] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:26 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:26.547830 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fae476-8735-4b74-b161-7347347ace3d None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:26.548333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fae476-8735-4b74-b161-7347347ace3d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:26.548815 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fae476-8735-4b74-b161-7347347ace3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:26.595032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27219bd2-40d3-4843-97a7-fe79daa6d5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:26.596751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27219bd2-40d3-4843-97a7-fe79daa6d5fd None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:26.597028 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-63fae476-8735-4b74-b161-7347347ace3d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:26.599410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-63fae476-8735-4b74-b161-7347347ace3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:26.603948 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27219bd2-40d3-4843-97a7-fe79daa6d5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:26.604820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27219bd2-40d3-4843-97a7-fe79daa6d5fd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:26.608651 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:26.649526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27219bd2-40d3-4843-97a7-fe79daa6d5fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:26.677773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-27219bd2-40d3-4843-97a7-fe79daa6d5fd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:49:26.677773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:26.677773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:26.677773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:26.677773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:26.677773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:26.677773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:26.677773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:26.677773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:26.677773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:26.677773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:26.677773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:26.677773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:26.677773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:26.677773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:26.677773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:26.678745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:26.678745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:26.678745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:26.678745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:26.678745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:26.678745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:26.678745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:26.678745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:26.678745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:26.678745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:26.678745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:26.678745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:26.678745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:26.678745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:26.678745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:26.678745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:26.679952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:26.679952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:26.679952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:26.679952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:26.679952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:26.679952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:26.679952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:26.679952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:26.679952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:26.679952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:26.679952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:49:26.679952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:26.688111 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 187/391] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:49:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:26.689991 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb906c-64b6-48cd-b484-52bfe03f9335 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:26.691240 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb906c-64b6-48cd-b484-52bfe03f9335 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:26.691717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb906c-64b6-48cd-b484-52bfe03f9335 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:26.702921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 188/392] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:26 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:26.704175 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb906c-64b6-48cd-b484-52bfe03f9335 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:26.783281 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 205/393] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:26.785091 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52691034-ff7b-4516-a313-2f5940a73628 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:26.787427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52691034-ff7b-4516-a313-2f5940a73628 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:26.788085 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52691034-ff7b-4516-a313-2f5940a73628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:26.824439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:26.827181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-52691034-ff7b-4516-a313-2f5940a73628 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:26.828552 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-52691034-ff7b-4516-a313-2f5940a73628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:26.981166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb906c-64b6-48cd-b484-52bfe03f9335 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:27.000520 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 206/394] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:27.015145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52691034-ff7b-4516-a313-2f5940a73628 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:27.101213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61695c-aecd-4240-838b-cf65544dfa18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:27.104701 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61695c-aecd-4240-838b-cf65544dfa18 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:27.106991 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61695c-aecd-4240-838b-cf65544dfa18 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:27.108175 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61695c-aecd-4240-838b-cf65544dfa18 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:27.111397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61695c-aecd-4240-838b-cf65544dfa18 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:27.112595 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61695c-aecd-4240-838b-cf65544dfa18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:27.118764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61695c-aecd-4240-838b-cf65544dfa18 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:27.140594 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 189/395] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:49:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:27.141643 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61695c-aecd-4240-838b-cf65544dfa18 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:27.256030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:27.334785 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:27.335734 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52691034-ff7b-4516-a313-2f5940a73628 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:27.340262 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61695c-aecd-4240-838b-cf65544dfa18 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:27.378351 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660a3ac-a3ad-4834-92e2-e2b89649283d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:27.379226 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660a3ac-a3ad-4834-92e2-e2b89649283d None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:27.380306 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660a3ac-a3ad-4834-92e2-e2b89649283d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:27.380306 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660a3ac-a3ad-4834-92e2-e2b89649283d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:27.393471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660a3ac-a3ad-4834-92e2-e2b89649283d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:27.402596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-6660a3ac-a3ad-4834-92e2-e2b89649283d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:27.402596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:27.402596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:27.402596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:27.402596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:27.402596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:27.402596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:27.402596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:27.402596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:27.402596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:27.402596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:27.402596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:27.402596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:27.402596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:27.402596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:27.402596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:27.404311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:27.404311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:27.404311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:27.404311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:27.404311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:27.404311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:27.404311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:27.404311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:27.404311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:27.404311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:27.404311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:27.404311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:27.404311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:27.404311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:27.404311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:27.404311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:27.405742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:27.405742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:27.405742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:27.405742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:27.405742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:27.405742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:27.405742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:27.405742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:27.405742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:27.405742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:27.405742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:27.405742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:27.407998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 190/396] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:27.409692 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660a3ac-a3ad-4834-92e2-e2b89649283d None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:27.417514 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303aa8e7-737d-4754-9146-7902b4af376b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:27.418979 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303aa8e7-737d-4754-9146-7902b4af376b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:27.419477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303aa8e7-737d-4754-9146-7902b4af376b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:27.419917 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303aa8e7-737d-4754-9146-7902b4af376b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:27.442523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303aa8e7-737d-4754-9146-7902b4af376b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:27.462497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-303aa8e7-737d-4754-9146-7902b4af376b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:49:27.462497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:27.462497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:27.462497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:27.462497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:27.462497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:27.462497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:27.462497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:27.462497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:27.462497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:27.462497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:27.462497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:27.462497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:27.462497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:27.462497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:27.462497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:27.463582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:27.463582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:27.463582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:27.463582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:27.463582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:27.463582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:27.463582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:27.463582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:27.463582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:27.463582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:27.463582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:27.463582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:27.463582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:27.463582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:27.463582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:27.463582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:27.464361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:27.464361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:27.464361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:27.464361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:27.464361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:27.464361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:27.464361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:27.464361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:27.464361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:27.464361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:27.464361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:49:27.464361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:27.464961 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 207/397] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:49:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 463 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:27.465810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303aa8e7-737d-4754-9146-7902b4af376b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:27.538640 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660a3ac-a3ad-4834-92e2-e2b89649283d None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:27.609878 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d0442-945a-4028-ae97-209ad23e5500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:27.611170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d0442-945a-4028-ae97-209ad23e5500 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:27.611683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d0442-945a-4028-ae97-209ad23e5500 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:27.612207 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d0442-945a-4028-ae97-209ad23e5500 None admin] PATH_INFO: `/v3/groups/f99848359a454a4abf52ac354138df49` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:27.613756 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d0442-945a-4028-ae97-209ad23e5500 None admin] RBAC: Authorizing `identity:get_group(group_id=f99848359a454a4abf52ac354138df49)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:27.615463 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d0442-945a-4028-ae97-209ad23e5500 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:27.617411 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 191/398] 10.222.0.27 () {56 vars in 1180 bytes} [Sat Jul 16 14:49:27 2022] GET /identity/v3/groups/f99848359a454a4abf52ac354138df49 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:27.623029 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d0442-945a-4028-ae97-209ad23e5500 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:27.656546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:27.661598 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303aa8e7-737d-4754-9146-7902b4af376b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:27.739160 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:27.745466 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d0442-945a-4028-ae97-209ad23e5500 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:27.753127 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd1247-6c4b-4dd5-b2d2-5675bc9233ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:27.755858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd1247-6c4b-4dd5-b2d2-5675bc9233ea None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:27.755858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd1247-6c4b-4dd5-b2d2-5675bc9233ea None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:27.755858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd1247-6c4b-4dd5-b2d2-5675bc9233ea None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:27.757550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd1247-6c4b-4dd5-b2d2-5675bc9233ea None admin] RBAC: Adding query filter params (name=member) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:27.758645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd1247-6c4b-4dd5-b2d2-5675bc9233ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:27.762514 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd1247-6c4b-4dd5-b2d2-5675bc9233ea None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:27.786940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 208/399] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:49:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:27.787961 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd1247-6c4b-4dd5-b2d2-5675bc9233ea None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:27.798602 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98717832-21e4-4cf1-9406-08b97def0230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:27.799413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98717832-21e4-4cf1-9406-08b97def0230 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:27.799803 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98717832-21e4-4cf1-9406-08b97def0230 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:27.800161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98717832-21e4-4cf1-9406-08b97def0230 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:27.801051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98717832-21e4-4cf1-9406-08b97def0230 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:27.801425 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98717832-21e4-4cf1-9406-08b97def0230 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:27.803044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98717832-21e4-4cf1-9406-08b97def0230 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:27.821550 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 192/400] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:49:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:27.824116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98717832-21e4-4cf1-9406-08b97def0230 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:27.964170 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd1247-6c4b-4dd5-b2d2-5675bc9233ea None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:27.983267 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:27.994676 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98717832-21e4-4cf1-9406-08b97def0230 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:28.035211 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfaf11-9614-425c-92bd-39d160a17b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:28.036491 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfaf11-9614-425c-92bd-39d160a17b19 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:28.037467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfaf11-9614-425c-92bd-39d160a17b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:28.038031 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddfaf11-9614-425c-92bd-39d160a17b19 None admin] PATH_INFO: `/v3/groups/f99848359a454a4abf52ac354138df49` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:28.041107 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb039067-bc85-4049-af2b-9f2c687021dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:28.042023 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb039067-bc85-4049-af2b-9f2c687021dd None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:28.042299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb039067-bc85-4049-af2b-9f2c687021dd None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:28.042583 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb039067-bc85-4049-af2b-9f2c687021dd None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:28.043901 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb039067-bc85-4049-af2b-9f2c687021dd None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:28.044558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfaf11-9614-425c-92bd-39d160a17b19 None admin] RBAC: Authorizing `identity:get_group(group_id=f99848359a454a4abf52ac354138df49)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:28.045394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb039067-bc85-4049-af2b-9f2c687021dd None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:28.048056 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddfaf11-9614-425c-92bd-39d160a17b19 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:28.051623 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 193/401] 10.222.0.27 () {56 vars in 1180 bytes} [Sat Jul 16 14:49:27 2022] GET /identity/v3/groups/f99848359a454a4abf52ac354138df49 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:28.052405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 209/402] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:27 2022] GET /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:28.053350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb039067-bc85-4049-af2b-9f2c687021dd None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:28.054007 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfaf11-9614-425c-92bd-39d160a17b19 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:28.145665 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:28.213799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb039067-bc85-4049-af2b-9f2c687021dd None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:28.245260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddfaf11-9614-425c-92bd-39d160a17b19 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:28.284196 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7775d-dedc-4b97-be59-f618c409ae95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:28.285071 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7775d-dedc-4b97-be59-f618c409ae95 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:28.285352 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7775d-dedc-4b97-be59-f618c409ae95 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:28.285630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7775d-dedc-4b97-be59-f618c409ae95 None admin] PATH_INFO: `/v3/groups/f99848359a454a4abf52ac354138df49` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:28.287007 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7775d-dedc-4b97-be59-f618c409ae95 None admin] RBAC: Authorizing `identity:get_group(group_id=f99848359a454a4abf52ac354138df49)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:28.287962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d7f87-d687-4e75-8f94-06016374aad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:28.288650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7775d-dedc-4b97-be59-f618c409ae95 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:28.291207 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d7f87-d687-4e75-8f94-06016374aad6 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:28.291379 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 194/403] 10.222.0.27 () {56 vars in 1180 bytes} [Sat Jul 16 14:49:28 2022] GET /identity/v3/groups/f99848359a454a4abf52ac354138df49 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:28.292127 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d7f87-d687-4e75-8f94-06016374aad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:28.292300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7775d-dedc-4b97-be59-f618c409ae95 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:28.293045 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d7f87-d687-4e75-8f94-06016374aad6 None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532/groups/f99848359a454a4abf52ac354138df49/roles/af11cd5265d14ac7902b1fb3afcc4be7` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:28.298273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d7f87-d687-4e75-8f94-06016374aad6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532, group_id=f99848359a454a4abf52ac354138df49, role_id=af11cd5265d14ac7902b1fb3afcc4be7)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:28.301432 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d7f87-d687-4e75-8f94-06016374aad6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:28.324449 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:28.387021 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 210/404] 10.222.0.27 () {58 vars in 1442 bytes} [Sat Jul 16 14:49:28 2022] PUT /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532/groups/f99848359a454a4abf52ac354138df49/roles/af11cd5265d14ac7902b1fb3afcc4be7 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:49:28.389383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d7f87-d687-4e75-8f94-06016374aad6 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:28.514868 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7775d-dedc-4b97-be59-f618c409ae95 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:28.592979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb72267-4955-4173-9fc2-75b060790618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:28.594521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb72267-4955-4173-9fc2-75b060790618 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:28.595138 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb72267-4955-4173-9fc2-75b060790618 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:28.595633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb72267-4955-4173-9fc2-75b060790618 None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:28.598476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb72267-4955-4173-9fc2-75b060790618 None admin] RBAC: Authorizing `identity:get_project(project_id=aad3af41314e48b4a48d335366f333ce)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:28.601329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb72267-4955-4173-9fc2-75b060790618 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:28.607036 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 195/405] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:28 2022] GET /identity/v3/projects/aad3af41314e48b4a48d335366f333ce => generated 302 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:28.611014 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eee9b9-9421-44ad-9755-cb89b9b85f97 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:28.611833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eee9b9-9421-44ad-9755-cb89b9b85f97 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:28.612555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eee9b9-9421-44ad-9755-cb89b9b85f97 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:28.615273 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 196/406] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:28.616127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0dc13-6f7d-411a-81f8-1555e31ca549 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:28.616418 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0dc13-6f7d-411a-81f8-1555e31ca549 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:28.616857 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0dc13-6f7d-411a-81f8-1555e31ca549 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:28.618925 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 197/407] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:28.619758 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0dc13-6f7d-411a-81f8-1555e31ca549 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:28.638492 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d7f87-d687-4e75-8f94-06016374aad6 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:28.671293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:28.688112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8fe18-9c5b-4bf0-a60b-8b79d1a6a20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:28.688998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8fe18-9c5b-4bf0-a60b-8b79d1a6a20a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:28.689290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8fe18-9c5b-4bf0-a60b-8b79d1a6a20a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:28.689556 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8fe18-9c5b-4bf0-a60b-8b79d1a6a20a None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:28.690825 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8fe18-9c5b-4bf0-a60b-8b79d1a6a20a None admin] RBAC: Authorizing `identity:get_project(project_id=aad3af41314e48b4a48d335366f333ce)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:28.692459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8fe18-9c5b-4bf0-a60b-8b79d1a6a20a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:28.699927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 211/408] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:28 2022] GET /identity/v3/projects/aad3af41314e48b4a48d335366f333ce => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:28.699927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66bcc4-36a5-46f2-8059-bb993b9c62a9 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:28.699927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66bcc4-36a5-46f2-8059-bb993b9c62a9 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:28.699927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66bcc4-36a5-46f2-8059-bb993b9c62a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:28.720717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ed66bcc4-36a5-46f2-8059-bb993b9c62a9 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:28.722254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ed66bcc4-36a5-46f2-8059-bb993b9c62a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:28.749746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:28.761985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0dc13-6f7d-411a-81f8-1555e31ca549 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:28.815161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 212/409] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:28.815161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c2775-3d26-4c8f-8ae5-15af2a0d306e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:28.815161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c2775-3d26-4c8f-8ae5-15af2a0d306e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:28.815161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c2775-3d26-4c8f-8ae5-15af2a0d306e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:28.818878 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eece11-eda1-40c6-b535-e9bacc059596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:28.819901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eece11-eda1-40c6-b535-e9bacc059596 None admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:28.820188 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eece11-eda1-40c6-b535-e9bacc059596 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:28.820467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eece11-eda1-40c6-b535-e9bacc059596 None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce/groups/f99848359a454a4abf52ac354138df49/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:28.824336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eece11-eda1-40c6-b535-e9bacc059596 None admin] RBAC: Authorizing `identity:create_grant(project_id=aad3af41314e48b4a48d335366f333ce, group_id=f99848359a454a4abf52ac354138df49, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:28.825937 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eece11-eda1-40c6-b535-e9bacc059596 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:28.844022 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-789c2775-3d26-4c8f-8ae5-15af2a0d306e None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:28.846017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-789c2775-3d26-4c8f-8ae5-15af2a0d306e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:28.913827 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 198/410] 10.222.0.27 () {58 vars in 1442 bytes} [Sat Jul 16 14:49:28 2022] PUT /identity/v3/projects/aad3af41314e48b4a48d335366f333ce/groups/f99848359a454a4abf52ac354138df49/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:49:28.915718 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eece11-eda1-40c6-b535-e9bacc059596 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:28.953520 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 213/411] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:28.955321 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c2775-3d26-4c8f-8ae5-15af2a0d306e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:29.054800 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:29.081193 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eece11-eda1-40c6-b535-e9bacc059596 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:29.123524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0755e99-0256-4da4-9bcc-b35d7ce2a531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:29.124528 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0755e99-0256-4da4-9bcc-b35d7ce2a531 None admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:29.124862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0755e99-0256-4da4-9bcc-b35d7ce2a531 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:29.125198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0755e99-0256-4da4-9bcc-b35d7ce2a531 None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce/groups/f99848359a454a4abf52ac354138df49/roles/af11cd5265d14ac7902b1fb3afcc4be7` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:29.127734 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0755e99-0256-4da4-9bcc-b35d7ce2a531 None admin] RBAC: Authorizing `identity:create_grant(project_id=aad3af41314e48b4a48d335366f333ce, group_id=f99848359a454a4abf52ac354138df49, role_id=af11cd5265d14ac7902b1fb3afcc4be7)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:29.129401 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0755e99-0256-4da4-9bcc-b35d7ce2a531 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:29.162366 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c2775-3d26-4c8f-8ae5-15af2a0d306e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:29.213260 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0488c-4275-4d6b-96f8-46311ae3982e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:29.214653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0488c-4275-4d6b-96f8-46311ae3982e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:29.215181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0488c-4275-4d6b-96f8-46311ae3982e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:29.215740 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0488c-4275-4d6b-96f8-46311ae3982e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:29.216208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 199/412] 10.222.0.27 () {58 vars in 1442 bytes} [Sat Jul 16 14:49:28 2022] PUT /identity/v3/projects/aad3af41314e48b4a48d335366f333ce/groups/f99848359a454a4abf52ac354138df49/roles/af11cd5265d14ac7902b1fb3afcc4be7 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:49:29.217832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0755e99-0256-4da4-9bcc-b35d7ce2a531 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:29.236818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0488c-4275-4d6b-96f8-46311ae3982e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:29.251977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-91c0488c-4275-4d6b-96f8-46311ae3982e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:49:29.251977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:29.251977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:29.251977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:29.251977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:29.251977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:29.251977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:29.251977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:29.251977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:29.251977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:29.251977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:29.251977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:29.251977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:29.251977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:29.251977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:29.251977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:29.253697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:29.253697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:29.253697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:29.253697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:29.253697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:29.253697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:29.253697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:29.253697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:29.253697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:29.253697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:29.253697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:29.253697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:29.253697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:29.253697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:29.253697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:29.253697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:29.255273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:29.255273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:29.255273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:29.255273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:29.255273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:29.255273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:29.255273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:29.255273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:29.255273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:29.255273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:29.255273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:49:29.255273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:29.255273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 214/413] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:49:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:29.265241 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0488c-4275-4d6b-96f8-46311ae3982e None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:29.277168 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:29.457873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0755e99-0256-4da4-9bcc-b35d7ce2a531 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:29.468457 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0488c-4275-4d6b-96f8-46311ae3982e None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:29.487538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:29.528597 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc031901-43f3-4aea-b0ad-50e7eef70acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:29.530555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc031901-43f3-4aea-b0ad-50e7eef70acc None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:29.531449 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc031901-43f3-4aea-b0ad-50e7eef70acc None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:29.532179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc031901-43f3-4aea-b0ad-50e7eef70acc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:29.537323 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29c09e-17dd-4c19-8533-a140058b86e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:29.538415 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29c09e-17dd-4c19-8533-a140058b86e9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:29.539844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29c09e-17dd-4c19-8533-a140058b86e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:29.539844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29c09e-17dd-4c19-8533-a140058b86e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:29.540957 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29c09e-17dd-4c19-8533-a140058b86e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:29.541490 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29c09e-17dd-4c19-8533-a140058b86e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:29.544014 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff29c09e-17dd-4c19-8533-a140058b86e9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:29.557012 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc031901-43f3-4aea-b0ad-50e7eef70acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:29.561062 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 215/414] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:49:29.570194 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29c09e-17dd-4c19-8533-a140058b86e9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:29.574361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-fc031901-43f3-4aea-b0ad-50e7eef70acc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:29.574361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:29.574361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:29.574361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:29.574361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:29.574361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:29.574361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:29.574361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:29.574361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:29.574361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:29.574361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:29.574361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:29.574361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:29.574361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:29.574361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:29.574361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:29.576145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:29.576145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:29.576145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:29.576145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:29.576145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:29.576145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:29.576145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:29.576145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:29.576145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:29.576145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:29.576145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:29.576145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:29.576145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:29.576145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:29.576145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:29.576145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:29.577787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:29.577787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:29.577787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:29.577787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:29.577787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:29.577787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:29.577787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:29.577787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:29.577787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:29.577787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:29.577787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:29.577787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:29.584155 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 200/415] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:29.593816 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc031901-43f3-4aea-b0ad-50e7eef70acc None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:29.702926 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29c09e-17dd-4c19-8533-a140058b86e9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:29.708931 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:29.750058 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a048bd-8bc3-43d8-a165-e095c5126357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:29.751462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a048bd-8bc3-43d8-a165-e095c5126357 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:29.751959 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a048bd-8bc3-43d8-a165-e095c5126357 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:29.752552 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a048bd-8bc3-43d8-a165-e095c5126357 None admin] PATH_INFO: `/v3/groups/dcdb8477e03e41fda3c3739065ecc097` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:29.754113 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a048bd-8bc3-43d8-a165-e095c5126357 None admin] RBAC: Authorizing `identity:get_group(group_id=dcdb8477e03e41fda3c3739065ecc097)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:29.756011 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a048bd-8bc3-43d8-a165-e095c5126357 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:29.758489 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 216/416] 10.222.0.27 () {56 vars in 1180 bytes} [Sat Jul 16 14:49:29 2022] GET /identity/v3/groups/dcdb8477e03e41fda3c3739065ecc097 => generated 229 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:49:29.764698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a048bd-8bc3-43d8-a165-e095c5126357 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:29.791855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc031901-43f3-4aea-b0ad-50e7eef70acc None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:29.816797 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:29.896201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a048bd-8bc3-43d8-a165-e095c5126357 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:29.909916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ce162-3608-42a7-b734-ced8a544ab27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:29.911014 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ce162-3608-42a7-b734-ced8a544ab27 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:29.911475 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ce162-3608-42a7-b734-ced8a544ab27 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:29.911941 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ce162-3608-42a7-b734-ced8a544ab27 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:29.913916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2ce162-3608-42a7-b734-ced8a544ab27 None admin] RBAC: Adding query filter params (name=member) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:29.913916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2ce162-3608-42a7-b734-ced8a544ab27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:29.923983 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2ce162-3608-42a7-b734-ced8a544ab27 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:29.942349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 201/417] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:49:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:49:29.948785 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ce162-3608-42a7-b734-ced8a544ab27 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:29.963419 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc9e2d-ce3e-49e7-9dc4-bd4bb4346909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:29.964947 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc9e2d-ce3e-49e7-9dc4-bd4bb4346909 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:29.965446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc9e2d-ce3e-49e7-9dc4-bd4bb4346909 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:29.965929 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc9e2d-ce3e-49e7-9dc4-bd4bb4346909 None admin] PATH_INFO: `/v3/projects/094ab8f948f540f7a205e832cbae4778` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:29.968280 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc9e2d-ce3e-49e7-9dc4-bd4bb4346909 None admin] RBAC: Authorizing `identity:get_project(project_id=094ab8f948f540f7a205e832cbae4778)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:29.970816 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc9e2d-ce3e-49e7-9dc4-bd4bb4346909 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:29.987823 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 217/418] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:29 2022] GET /identity/v3/projects/094ab8f948f540f7a205e832cbae4778 => generated 344 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:49:29.995163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc9e2d-ce3e-49e7-9dc4-bd4bb4346909 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:30.206697 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ce162-3608-42a7-b734-ced8a544ab27 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:30.329030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc9e2d-ce3e-49e7-9dc4-bd4bb4346909 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:30.332550 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e648db-6554-41a7-ac40-84ede57976d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:30.333471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e648db-6554-41a7-ac40-84ede57976d0 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:30.333970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e648db-6554-41a7-ac40-84ede57976d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:30.334462 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e648db-6554-41a7-ac40-84ede57976d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:30.348911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:30.357838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:30.443345 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc3156-5783-4dac-9131-f536ea38aedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:30.444273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc3156-5783-4dac-9131-f536ea38aedc None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:30.444555 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc3156-5783-4dac-9131-f536ea38aedc None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:30.444831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc3156-5783-4dac-9131-f536ea38aedc None admin] PATH_INFO: `/v3/projects/094ab8f948f540f7a205e832cbae4778/groups/dcdb8477e03e41fda3c3739065ecc097/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:30.451910 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc3156-5783-4dac-9131-f536ea38aedc None admin] RBAC: Authorizing `identity:create_grant(project_id=094ab8f948f540f7a205e832cbae4778, group_id=dcdb8477e03e41fda3c3739065ecc097, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:30.455842 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc3156-5783-4dac-9131-f536ea38aedc None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:30.505483 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e648db-6554-41a7-ac40-84ede57976d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:30.507182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e648db-6554-41a7-ac40-84ede57976d0 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:30.607124 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 218/419] 10.222.0.27 () {58 vars in 1442 bytes} [Sat Jul 16 14:49:29 2022] PUT /identity/v3/projects/094ab8f948f540f7a205e832cbae4778/groups/dcdb8477e03e41fda3c3739065ecc097/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 14:49:30.635230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 202/420] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:49:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:49:30.641157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc3156-5783-4dac-9131-f536ea38aedc None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:30.808801 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc3156-5783-4dac-9131-f536ea38aedc None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:30.874000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f062a3a-2ebc-41fc-8437-b13054139a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:30.875521 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f062a3a-2ebc-41fc-8437-b13054139a93 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:30.876017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f062a3a-2ebc-41fc-8437-b13054139a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:30.876581 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f062a3a-2ebc-41fc-8437-b13054139a93 None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:30.878850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f062a3a-2ebc-41fc-8437-b13054139a93 None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:30.881681 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f062a3a-2ebc-41fc-8437-b13054139a93 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:30.886466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 219/421] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:30 2022] GET /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:49:30.897741 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e648db-6554-41a7-ac40-84ede57976d0 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:30.955293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:31.053360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e648db-6554-41a7-ac40-84ede57976d0 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:31.097580 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d095a-7926-487b-8ccf-8fa78c7c61c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:31.098454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d095a-7926-487b-8ccf-8fa78c7c61c6 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:31.098737 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d095a-7926-487b-8ccf-8fa78c7c61c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:31.099036 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d095a-7926-487b-8ccf-8fa78c7c61c6 None admin] PATH_INFO: `/v3/groups/f99848359a454a4abf52ac354138df49` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:31.100251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d095a-7926-487b-8ccf-8fa78c7c61c6 None admin] RBAC: Authorizing `identity:get_group(group_id=f99848359a454a4abf52ac354138df49)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:31.101780 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d095a-7926-487b-8ccf-8fa78c7c61c6 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:31.104155 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 203/422] 10.222.0.27 () {56 vars in 1180 bytes} [Sat Jul 16 14:49:30 2022] GET /identity/v3/groups/f99848359a454a4abf52ac354138df49 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:31.110842 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f062a3a-2ebc-41fc-8437-b13054139a93 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:31.212160 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:31.234459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8fa7b-5111-4a7a-a9ec-c1982ebac7ff None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:31.234822 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8fa7b-5111-4a7a-a9ec-c1982ebac7ff None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:31.235132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8fa7b-5111-4a7a-a9ec-c1982ebac7ff None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:31.236734 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 204/423] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:31.248408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f8639-3f5d-4011-afd3-4d2521942109 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:31.248408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f8639-3f5d-4011-afd3-4d2521942109 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:31.248408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f8639-3f5d-4011-afd3-4d2521942109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:31.293336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-bc8f8639-3f5d-4011-afd3-4d2521942109 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:31.295408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-bc8f8639-3f5d-4011-afd3-4d2521942109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:31.361118 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f062a3a-2ebc-41fc-8437-b13054139a93 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:31.444787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 205/424] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:31.459379 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9b399-dd89-4182-b89f-be7a7af6ed87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:31.460538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9b399-dd89-4182-b89f-be7a7af6ed87 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:31.461069 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9b399-dd89-4182-b89f-be7a7af6ed87 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:31.462454 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9b399-dd89-4182-b89f-be7a7af6ed87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:31.464085 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9b399-dd89-4182-b89f-be7a7af6ed87 None admin] RBAC: Adding query filter params (group.id=f99848359a454a4abf52ac354138df49, role.id=b7f92100b08f4dc6b2b6866a2955d2ab, scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:31.464626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9b399-dd89-4182-b89f-be7a7af6ed87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:31.467317 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9b399-dd89-4182-b89f-be7a7af6ed87 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:31.469470 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f8639-3f5d-4011-afd3-4d2521942109 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:31.501177 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 220/425] 10.222.0.27 () {56 vars in 1376 bytes} [Sat Jul 16 14:49:31 2022] GET /identity/v3/role_assignments?group.id=f99848359a454a4abf52ac354138df49&scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532&role.id=b7f92100b08f4dc6b2b6866a2955d2ab => generated 611 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:31.547882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:31.667101 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f8639-3f5d-4011-afd3-4d2521942109 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:31.720676 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f9a06-4db8-4019-8929-b66c1504a7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:31.721441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f9a06-4db8-4019-8929-b66c1504a7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:31.721724 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f9a06-4db8-4019-8929-b66c1504a7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:31.721998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f9a06-4db8-4019-8929-b66c1504a7c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:31.749298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f9a06-4db8-4019-8929-b66c1504a7c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:31.757731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-745f9a06-4db8-4019-8929-b66c1504a7c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:49:31.757731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:31.757731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:31.757731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:31.757731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:31.757731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:31.757731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:31.757731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:31.757731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:31.757731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:31.757731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:31.757731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:31.757731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:31.757731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:31.757731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:31.757731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:31.759251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:31.759251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:31.759251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:31.759251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:31.759251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:31.759251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:31.759251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:31.759251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:31.759251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:31.759251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:31.759251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:31.759251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:31.759251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:31.759251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:31.759251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:31.759251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:31.760314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:31.760314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:31.760314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:31.760314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:31.760314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:31.760314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:31.760314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:31.760314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:31.760314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:31.760314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:31.760314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:49:31.760314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:31.761599 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 206/426] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:49:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:31.769441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9b399-dd89-4182-b89f-be7a7af6ed87 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:31.886486 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:31.890907 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9b399-dd89-4182-b89f-be7a7af6ed87 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:31.929753 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3a267-9582-4004-b603-85f0b198d6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:31.931404 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3a267-9582-4004-b603-85f0b198d6ac None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:31.932252 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3a267-9582-4004-b603-85f0b198d6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:31.932530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3a267-9582-4004-b603-85f0b198d6ac None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:31.933303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3a267-9582-4004-b603-85f0b198d6ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:31.933614 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3a267-9582-4004-b603-85f0b198d6ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:31.935038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3a267-9582-4004-b603-85f0b198d6ac None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:31.944626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 221/427] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:49:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:31.950784 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f9a06-4db8-4019-8929-b66c1504a7c1 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:32.066566 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:32.077132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f9a06-4db8-4019-8929-b66c1504a7c1 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:32.138445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e864c-cf75-4897-a94e-d5f98f5c82bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:32.139185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e864c-cf75-4897-a94e-d5f98f5c82bc None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:32.139517 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e864c-cf75-4897-a94e-d5f98f5c82bc None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:32.139820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e864c-cf75-4897-a94e-d5f98f5c82bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:32.220681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769e864c-cf75-4897-a94e-d5f98f5c82bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:32.222196 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769e864c-cf75-4897-a94e-d5f98f5c82bc None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:32.339366 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:32.347955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 207/428] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:49:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 16 14:49:32.356482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3a267-9582-4004-b603-85f0b198d6ac None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:32.405883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0947a-033c-4a9d-9359-11b18c5937c8 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:32.406516 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0947a-033c-4a9d-9359-11b18c5937c8 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:32.407159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0947a-033c-4a9d-9359-11b18c5937c8 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:32.409124 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 208/429] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:49:32.416136 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f754f16-615a-4855-a233-8e2f48d20f8c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:32.416759 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f754f16-615a-4855-a233-8e2f48d20f8c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:32.417239 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f754f16-615a-4855-a233-8e2f48d20f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:32.440923 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-9f754f16-615a-4855-a233-8e2f48d20f8c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:32.442591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-9f754f16-615a-4855-a233-8e2f48d20f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:32.583046 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 209/430] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 14:49:32.591699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3a267-9582-4004-b603-85f0b198d6ac None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:32.607424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f754f16-615a-4855-a233-8e2f48d20f8c None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:32.672170 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d75055-ac98-4d13-b5f6-8779d2da5615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:32.673696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d75055-ac98-4d13-b5f6-8779d2da5615 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:32.674194 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d75055-ac98-4d13-b5f6-8779d2da5615 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:32.674692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d75055-ac98-4d13-b5f6-8779d2da5615 None admin] PATH_INFO: `/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:32.677068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d75055-ac98-4d13-b5f6-8779d2da5615 None admin] RBAC: Authorizing `identity:get_project(project_id=e7dc9e4cebb9431d8fc4a051f9cd5532)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:32.680580 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d75055-ac98-4d13-b5f6-8779d2da5615 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:32.684285 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 222/431] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:32 2022] GET /identity/v3/projects/e7dc9e4cebb9431d8fc4a051f9cd5532 => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:32.691086 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d75055-ac98-4d13-b5f6-8779d2da5615 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:32.750614 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:32.800129 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f754f16-615a-4855-a233-8e2f48d20f8c None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:32.844682 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:32.863487 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44700a6e-dc1a-4c6a-8a4f-433a3b7314cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:32.865372 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44700a6e-dc1a-4c6a-8a4f-433a3b7314cc None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:32.865848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44700a6e-dc1a-4c6a-8a4f-433a3b7314cc None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:32.866318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44700a6e-dc1a-4c6a-8a4f-433a3b7314cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:32.884406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d75055-ac98-4d13-b5f6-8779d2da5615 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:32.894357 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44700a6e-dc1a-4c6a-8a4f-433a3b7314cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:32.912281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-44700a6e-dc1a-4c6a-8a4f-433a3b7314cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:32.912281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:32.912281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:32.912281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:32.912281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:32.912281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:32.912281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:32.912281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:32.912281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:32.912281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:32.912281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:32.912281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:32.912281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:32.912281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:32.912281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:32.912281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:32.914079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:32.914079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:32.914079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:32.914079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:32.914079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:32.914079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:32.914079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:32.914079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:32.914079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:32.914079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:32.914079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:32.914079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:32.914079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:32.914079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:32.914079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:32.914079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:32.916112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:32.916112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:32.916112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:32.916112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:32.916112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:32.916112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:32.916112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:32.916112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:32.916112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:32.916112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:32.916112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 16 14:49:32.916112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:32.916112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 210/432] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:32.921883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44700a6e-dc1a-4c6a-8a4f-433a3b7314cc None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:32.957397 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae079f-7475-40e9-873b-1bf076362d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:32.959314 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae079f-7475-40e9-873b-1bf076362d4f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:32.960040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae079f-7475-40e9-873b-1bf076362d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:32.960736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae079f-7475-40e9-873b-1bf076362d4f None admin] PATH_INFO: `/v3/groups/f99848359a454a4abf52ac354138df49` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:32.963694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae079f-7475-40e9-873b-1bf076362d4f None admin] RBAC: Authorizing `identity:get_group(group_id=f99848359a454a4abf52ac354138df49)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:32.966572 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae079f-7475-40e9-873b-1bf076362d4f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:32.970407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 223/433] 10.222.0.27 () {56 vars in 1180 bytes} [Sat Jul 16 14:49:32 2022] GET /identity/v3/groups/f99848359a454a4abf52ac354138df49 => generated 226 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:32.980042 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae079f-7475-40e9-873b-1bf076362d4f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:33.087861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44700a6e-dc1a-4c6a-8a4f-433a3b7314cc None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:33.132502 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01f21b-1970-4e0b-8d0b-14b0a5570a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:33.133750 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01f21b-1970-4e0b-8d0b-14b0a5570a2f None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:33.134036 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01f21b-1970-4e0b-8d0b-14b0a5570a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:33.134310 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01f21b-1970-4e0b-8d0b-14b0a5570a2f None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:33.135139 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01f21b-1970-4e0b-8d0b-14b0a5570a2f None admin] RBAC: Adding query filter params (name=member) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:33.135448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01f21b-1970-4e0b-8d0b-14b0a5570a2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:33.136904 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01f21b-1970-4e0b-8d0b-14b0a5570a2f None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:33.153669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 211/434] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:49:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:33.155484 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01f21b-1970-4e0b-8d0b-14b0a5570a2f None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:33.181866 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:33.193583 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae079f-7475-40e9-873b-1bf076362d4f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:33.217260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:33.266385 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f4355-7ba1-464d-a1f9-fbd3e338d642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:33.267920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f4355-7ba1-464d-a1f9-fbd3e338d642 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:33.268513 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f4355-7ba1-464d-a1f9-fbd3e338d642 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:33.269134 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f4355-7ba1-464d-a1f9-fbd3e338d642 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:33.270986 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f4355-7ba1-464d-a1f9-fbd3e338d642 None admin] RBAC: Adding query filter params (group.id=f99848359a454a4abf52ac354138df49, role.id=af11cd5265d14ac7902b1fb3afcc4be7, scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:33.271784 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f4355-7ba1-464d-a1f9-fbd3e338d642 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:33.275516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276f4355-7ba1-464d-a1f9-fbd3e338d642 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:33.283373 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01f21b-1970-4e0b-8d0b-14b0a5570a2f None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:33.294927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 224/435] 10.222.0.27 () {56 vars in 1376 bytes} [Sat Jul 16 14:49:32 2022] GET /identity/v3/role_assignments?group.id=f99848359a454a4abf52ac354138df49&scope.project.id=e7dc9e4cebb9431d8fc4a051f9cd5532&role.id=af11cd5265d14ac7902b1fb3afcc4be7 => generated 611 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:33.334383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56feb8ee-8c89-4eca-b675-dd384f8473c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:33.335086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56feb8ee-8c89-4eca-b675-dd384f8473c8 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:33.335367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56feb8ee-8c89-4eca-b675-dd384f8473c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:33.335637 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56feb8ee-8c89-4eca-b675-dd384f8473c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:33.339894 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b06fd-7b9a-40e2-9b6e-ecad0088b50e None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:33.340193 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b06fd-7b9a-40e2-9b6e-ecad0088b50e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:33.340439 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b06fd-7b9a-40e2-9b6e-ecad0088b50e None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:33.342287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 225/436] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:49:33.348525 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fbc6b-81a7-4724-bc78-f45b5107645a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:33.348964 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fbc6b-81a7-4724-bc78-f45b5107645a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:33.349337 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fbc6b-81a7-4724-bc78-f45b5107645a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:33.369018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a57fbc6b-81a7-4724-bc78-f45b5107645a None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:33.370390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a57fbc6b-81a7-4724-bc78-f45b5107645a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:33.483756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 226/437] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 14:49:33.500539 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56feb8ee-8c89-4eca-b675-dd384f8473c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:33.502372 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56feb8ee-8c89-4eca-b675-dd384f8473c8 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:33.508650 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fbc6b-81a7-4724-bc78-f45b5107645a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:33.588097 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:33.651518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:33.671573 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fbc6b-81a7-4724-bc78-f45b5107645a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:33.700240 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 212/438] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:49:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 16 14:49:33.708414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56feb8ee-8c89-4eca-b675-dd384f8473c8 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:33.739895 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5229a1-5200-4dbf-b7b9-7d9e4c95db17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:33.741244 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5229a1-5200-4dbf-b7b9-7d9e4c95db17 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:33.742341 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5229a1-5200-4dbf-b7b9-7d9e4c95db17 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:33.742341 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5229a1-5200-4dbf-b7b9-7d9e4c95db17 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:33.763208 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5229a1-5200-4dbf-b7b9-7d9e4c95db17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:33.771645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-0a5229a1-5200-4dbf-b7b9-7d9e4c95db17 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:49:33.771645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:33.771645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:33.771645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:33.771645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:33.771645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:33.771645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:33.771645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:33.771645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:33.771645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:33.771645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:33.771645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:33.771645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:33.771645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:33.771645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:33.771645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:33.773412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:33.773412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:33.773412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:33.773412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:33.773412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:33.773412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:33.773412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:33.773412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:33.773412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:33.773412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:33.773412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:33.773412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:33.773412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:33.773412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:33.773412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:33.773412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:33.776000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:33.776000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:33.776000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:33.776000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:33.776000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:33.776000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:33.776000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:33.776000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:33.776000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:33.776000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:33.776000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:49:33.776000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:33.779085 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 227/439] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:49:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:49:33.781008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7e8b5-9897-4283-b4f2-dd8856e547a1 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:33.781354 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7e8b5-9897-4283-b4f2-dd8856e547a1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:33.781638 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7e8b5-9897-4283-b4f2-dd8856e547a1 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:33.783301 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 228/440] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:33.783996 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7e8b5-9897-4283-b4f2-dd8856e547a1 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:33.856548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56feb8ee-8c89-4eca-b675-dd384f8473c8 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:33.901634 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e1369-a8c2-4095-bcad-adf4486a864d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:33.902372 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e1369-a8c2-4095-bcad-adf4486a864d None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:33.902657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e1369-a8c2-4095-bcad-adf4486a864d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:33.902968 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e1369-a8c2-4095-bcad-adf4486a864d None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:33.904229 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e1369-a8c2-4095-bcad-adf4486a864d None admin] RBAC: Authorizing `identity:get_project(project_id=aad3af41314e48b4a48d335366f333ce)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:33.905729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9e1369-a8c2-4095-bcad-adf4486a864d None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:33.907521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 213/441] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:33 2022] GET /identity/v3/projects/aad3af41314e48b4a48d335366f333ce => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:33.912583 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde2b2c-22ca-4b1c-82f1-2088adaf30cb None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:33.912870 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde2b2c-22ca-4b1c-82f1-2088adaf30cb None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:33.913145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde2b2c-22ca-4b1c-82f1-2088adaf30cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:33.931089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-dfde2b2c-22ca-4b1c-82f1-2088adaf30cb None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:33.932365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-dfde2b2c-22ca-4b1c-82f1-2088adaf30cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:33.938004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7e8b5-9897-4283-b4f2-dd8856e547a1 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:33.942541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:33.978088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa1b77-aa02-454a-a892-5b51b02e79da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:33.978759 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa1b77-aa02-454a-a892-5b51b02e79da None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:33.979074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa1b77-aa02-454a-a892-5b51b02e79da None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:33.979355 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa1b77-aa02-454a-a892-5b51b02e79da None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:33.980192 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa1b77-aa02-454a-a892-5b51b02e79da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:33.980501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa1b77-aa02-454a-a892-5b51b02e79da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:33.981889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa1b77-aa02-454a-a892-5b51b02e79da None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:33.990424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 229/442] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:49:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:33.991291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa1b77-aa02-454a-a892-5b51b02e79da None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:34.011993 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 214/443] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:34.013155 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde2b2c-22ca-4b1c-82f1-2088adaf30cb None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:34.026033 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:34.128775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa1b77-aa02-454a-a892-5b51b02e79da None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:34.130976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde2b2c-22ca-4b1c-82f1-2088adaf30cb None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:34.165974 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa73cf-66bb-4dca-83fd-92f56230fd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:34.166832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa73cf-66bb-4dca-83fd-92f56230fd0c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:34.167139 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa73cf-66bb-4dca-83fd-92f56230fd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:34.167412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa73cf-66bb-4dca-83fd-92f56230fd0c None admin] PATH_INFO: `/v3/groups/f99848359a454a4abf52ac354138df49` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:34.168671 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa73cf-66bb-4dca-83fd-92f56230fd0c None admin] RBAC: Authorizing `identity:get_group(group_id=f99848359a454a4abf52ac354138df49)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:34.170120 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa73cf-66bb-4dca-83fd-92f56230fd0c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:34.171790 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 230/444] 10.222.0.27 () {56 vars in 1180 bytes} [Sat Jul 16 14:49:33 2022] GET /identity/v3/groups/f99848359a454a4abf52ac354138df49 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:34.172562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa73cf-66bb-4dca-83fd-92f56230fd0c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:34.193843 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96e1a5-9889-4682-8c27-c893c9cbfecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:34.194580 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96e1a5-9889-4682-8c27-c893c9cbfecb None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:34.194867 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96e1a5-9889-4682-8c27-c893c9cbfecb None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:34.195177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96e1a5-9889-4682-8c27-c893c9cbfecb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:34.210878 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:34.260167 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:34.275269 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96e1a5-9889-4682-8c27-c893c9cbfecb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:34.276778 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96e1a5-9889-4682-8c27-c893c9cbfecb None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:34.280519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa73cf-66bb-4dca-83fd-92f56230fd0c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:34.334419 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccb086-9f06-4bc1-933b-0919baace421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:34.336082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccb086-9f06-4bc1-933b-0919baace421 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:34.336642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccb086-9f06-4bc1-933b-0919baace421 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:34.337407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccb086-9f06-4bc1-933b-0919baace421 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:34.355206 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccb086-9f06-4bc1-933b-0919baace421 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:34.363049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-78ccb086-9f06-4bc1-933b-0919baace421 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:49:34.363049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:34.363049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:34.363049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:34.363049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:34.363049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:34.363049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:34.363049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:34.363049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:34.363049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:34.363049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:34.363049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:34.363049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:34.363049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:34.363049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:34.363049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:34.364617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:34.364617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:34.364617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:34.364617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:34.364617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:34.364617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:34.364617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:34.364617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:34.364617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:34.364617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:34.364617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:34.364617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:34.364617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:34.364617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:34.364617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:34.364617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:34.365837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:34.365837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:34.365837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:34.365837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:34.365837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:34.365837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:34.365837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:34.365837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:34.365837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:34.365837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:34.365837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:49:34.365837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:34.365837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 231/445] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:49:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:34.365837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccb086-9f06-4bc1-933b-0919baace421 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:34.392077 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 215/446] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:49:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:49:34.392077 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96e1a5-9889-4682-8c27-c893c9cbfecb None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:34.443495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:34.478702 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccb086-9f06-4bc1-933b-0919baace421 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:34.510358 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96e1a5-9889-4682-8c27-c893c9cbfecb None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:34.514436 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0e7d2-955e-4bfe-b934-1917ff2eea35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:34.515142 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0e7d2-955e-4bfe-b934-1917ff2eea35 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:34.515464 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0e7d2-955e-4bfe-b934-1917ff2eea35 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:34.515693 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0e7d2-955e-4bfe-b934-1917ff2eea35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:34.516566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0e7d2-955e-4bfe-b934-1917ff2eea35 None admin] RBAC: Adding query filter params (group.id=f99848359a454a4abf52ac354138df49, role.id=b7f92100b08f4dc6b2b6866a2955d2ab, scope.project.id=aad3af41314e48b4a48d335366f333ce) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:34.517064 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0e7d2-955e-4bfe-b934-1917ff2eea35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:34.518394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0e7d2-955e-4bfe-b934-1917ff2eea35 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:34.526392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 232/447] 10.222.0.27 () {56 vars in 1376 bytes} [Sat Jul 16 14:49:34 2022] GET /identity/v3/role_assignments?group.id=f99848359a454a4abf52ac354138df49&scope.project.id=aad3af41314e48b4a48d335366f333ce&role.id=b7f92100b08f4dc6b2b6866a2955d2ab => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:34.527359 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0e7d2-955e-4bfe-b934-1917ff2eea35 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:34.535346 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:34.571978 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931a542-1d02-40d0-9b55-0d803373f78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:34.572724 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931a542-1d02-40d0-9b55-0d803373f78e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:34.573006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931a542-1d02-40d0-9b55-0d803373f78e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:34.573283 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931a542-1d02-40d0-9b55-0d803373f78e None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:34.574062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931a542-1d02-40d0-9b55-0d803373f78e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:34.574363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931a542-1d02-40d0-9b55-0d803373f78e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:34.576086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931a542-1d02-40d0-9b55-0d803373f78e None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:34.587733 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 216/448] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:34.594842 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931a542-1d02-40d0-9b55-0d803373f78e None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:34.685025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0e7d2-955e-4bfe-b934-1917ff2eea35 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:34.731916 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708d1a7-40c8-4bab-bd40-c7884535bbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:34.732798 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708d1a7-40c8-4bab-bd40-c7884535bbc3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:34.733076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708d1a7-40c8-4bab-bd40-c7884535bbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:34.733349 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708d1a7-40c8-4bab-bd40-c7884535bbc3 None admin] PATH_INFO: `/v3/projects/aad3af41314e48b4a48d335366f333ce` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:34.735334 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708d1a7-40c8-4bab-bd40-c7884535bbc3 None admin] RBAC: Authorizing `identity:get_project(project_id=aad3af41314e48b4a48d335366f333ce)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:34.737703 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708d1a7-40c8-4bab-bd40-c7884535bbc3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:34.742938 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 233/449] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:34 2022] GET /identity/v3/projects/aad3af41314e48b4a48d335366f333ce => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:34.748028 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708d1a7-40c8-4bab-bd40-c7884535bbc3 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:34.780988 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931a542-1d02-40d0-9b55-0d803373f78e None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:34.790685 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:34.838732 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1113db-1ad5-4ace-9bbb-ba8408ea0685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:34.839455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1113db-1ad5-4ace-9bbb-ba8408ea0685 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:34.839741 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1113db-1ad5-4ace-9bbb-ba8408ea0685 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:34.840016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1113db-1ad5-4ace-9bbb-ba8408ea0685 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:34.881101 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:34.937495 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708d1a7-40c8-4bab-bd40-c7884535bbc3 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:34.952341 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1113db-1ad5-4ace-9bbb-ba8408ea0685 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:34.953833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1113db-1ad5-4ace-9bbb-ba8408ea0685 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:35.001883 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c922cee-f1f2-479e-8d43-1e7c0163ed6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:35.002747 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c922cee-f1f2-479e-8d43-1e7c0163ed6a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:35.003053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c922cee-f1f2-479e-8d43-1e7c0163ed6a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:35.003332 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c922cee-f1f2-479e-8d43-1e7c0163ed6a None admin] PATH_INFO: `/v3/groups/f99848359a454a4abf52ac354138df49` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:35.004575 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c922cee-f1f2-479e-8d43-1e7c0163ed6a None admin] RBAC: Authorizing `identity:get_group(group_id=f99848359a454a4abf52ac354138df49)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:35.006030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c922cee-f1f2-479e-8d43-1e7c0163ed6a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:35.007745 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 234/450] 10.222.0.27 () {56 vars in 1180 bytes} [Sat Jul 16 14:49:34 2022] GET /identity/v3/groups/f99848359a454a4abf52ac354138df49 => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:49:35.017301 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c922cee-f1f2-479e-8d43-1e7c0163ed6a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:35.075050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 217/451] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:49:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 16 14:49:35.080659 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1113db-1ad5-4ace-9bbb-ba8408ea0685 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:35.090204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:35.147383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c922cee-f1f2-479e-8d43-1e7c0163ed6a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:35.190107 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c6f55-850e-409c-acfe-5a8f0d9fb9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:35.190766 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c6f55-850e-409c-acfe-5a8f0d9fb9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:35.191073 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c6f55-850e-409c-acfe-5a8f0d9fb9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:35.191337 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c6f55-850e-409c-acfe-5a8f0d9fb9b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:35.192186 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c6f55-850e-409c-acfe-5a8f0d9fb9b2 None admin] RBAC: Adding query filter params (group.id=f99848359a454a4abf52ac354138df49, role.id=af11cd5265d14ac7902b1fb3afcc4be7, scope.project.id=aad3af41314e48b4a48d335366f333ce) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:35.192490 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c6f55-850e-409c-acfe-5a8f0d9fb9b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:35.194052 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c6f55-850e-409c-acfe-5a8f0d9fb9b2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:35.200750 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 235/452] 10.222.0.27 () {56 vars in 1376 bytes} [Sat Jul 16 14:49:35 2022] GET /identity/v3/role_assignments?group.id=f99848359a454a4abf52ac354138df49&scope.project.id=aad3af41314e48b4a48d335366f333ce&role.id=af11cd5265d14ac7902b1fb3afcc4be7 => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:49:35.201064 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1113db-1ad5-4ace-9bbb-ba8408ea0685 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:35.246476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e62a99-6ffa-4ea4-acb7-d0beabf0928d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:35.247828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e62a99-6ffa-4ea4-acb7-d0beabf0928d None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:35.248112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e62a99-6ffa-4ea4-acb7-d0beabf0928d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:35.248432 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e62a99-6ffa-4ea4-acb7-d0beabf0928d None admin] PATH_INFO: `/v3/projects/094ab8f948f540f7a205e832cbae4778` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:35.249956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e62a99-6ffa-4ea4-acb7-d0beabf0928d None admin] RBAC: Authorizing `identity:get_project(project_id=094ab8f948f540f7a205e832cbae4778)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:35.252633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e62a99-6ffa-4ea4-acb7-d0beabf0928d None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:35.254979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 218/453] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:49:35 2022] GET /identity/v3/projects/094ab8f948f540f7a205e832cbae4778 => generated 344 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:49:35.261818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c6f55-850e-409c-acfe-5a8f0d9fb9b2 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:35.266698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:35.389899 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c6f55-850e-409c-acfe-5a8f0d9fb9b2 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:35.425537 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf61b76-279b-49d7-95d4-d7911e9e0ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:35.426395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf61b76-279b-49d7-95d4-d7911e9e0ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:35.426679 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf61b76-279b-49d7-95d4-d7911e9e0ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:35.426979 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf61b76-279b-49d7-95d4-d7911e9e0ef3 None admin] PATH_INFO: `/v3/groups/dcdb8477e03e41fda3c3739065ecc097` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:35.428175 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf61b76-279b-49d7-95d4-d7911e9e0ef3 None admin] RBAC: Authorizing `identity:get_group(group_id=dcdb8477e03e41fda3c3739065ecc097)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:35.429634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf61b76-279b-49d7-95d4-d7911e9e0ef3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:35.431353 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 236/454] 10.222.0.27 () {56 vars in 1180 bytes} [Sat Jul 16 14:49:35 2022] GET /identity/v3/groups/dcdb8477e03e41fda3c3739065ecc097 => generated 229 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:35.437259 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e62a99-6ffa-4ea4-acb7-d0beabf0928d None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:35.540704 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:35.566441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e62a99-6ffa-4ea4-acb7-d0beabf0928d None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:35.602475 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf41cbf-060d-4f01-beab-e7581ba35f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:35.603254 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf41cbf-060d-4f01-beab-e7581ba35f46 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:35.603438 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf41cbf-060d-4f01-beab-e7581ba35f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:35.603716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf41cbf-060d-4f01-beab-e7581ba35f46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:35.604560 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf41cbf-060d-4f01-beab-e7581ba35f46 None admin] RBAC: Adding query filter params (group.id=dcdb8477e03e41fda3c3739065ecc097, role.id=0efeff46573247f3b9f1e2dec2968672, scope.project.id=094ab8f948f540f7a205e832cbae4778) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:35.604874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf41cbf-060d-4f01-beab-e7581ba35f46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:35.606314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf41cbf-060d-4f01-beab-e7581ba35f46 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:35.612211 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 219/455] 10.222.0.27 () {56 vars in 1376 bytes} [Sat Jul 16 14:49:35 2022] GET /identity/v3/role_assignments?group.id=dcdb8477e03e41fda3c3739065ecc097&scope.project.id=094ab8f948f540f7a205e832cbae4778&role.id=0efeff46573247f3b9f1e2dec2968672 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:38.632072 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de9f2b-c8ff-4f79-aa47-006f81d3f69b None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:38.632858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de9f2b-c8ff-4f79-aa47-006f81d3f69b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:38.632858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de9f2b-c8ff-4f79-aa47-006f81d3f69b None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:38.635513 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 237/456] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:38.644312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606f629-646c-4afd-aaf6-27eb9905fc5d None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:38.645202 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606f629-646c-4afd-aaf6-27eb9905fc5d None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:38.645429 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606f629-646c-4afd-aaf6-27eb9905fc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:38.700209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-d606f629-646c-4afd-aaf6-27eb9905fc5d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:38.702252 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-d606f629-646c-4afd-aaf6-27eb9905fc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:38.883649 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 220/457] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 14:49:38.944776 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de9f2b-c8ff-4f79-aa47-006f81d3f69b None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:39.052027 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:39.100688 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de9f2b-c8ff-4f79-aa47-006f81d3f69b None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:39.156092 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7bf89-b0d5-4e39-83e2-c80c98252f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:39.156838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7bf89-b0d5-4e39-83e2-c80c98252f79 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:39.157125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7bf89-b0d5-4e39-83e2-c80c98252f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:39.157406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7bf89-b0d5-4e39-83e2-c80c98252f79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:39.181303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7bf89-b0d5-4e39-83e2-c80c98252f79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:39.182830 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7bf89-b0d5-4e39-83e2-c80c98252f79 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:39.209229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-aca7bf89-b0d5-4e39-83e2-c80c98252f79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:39.209229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:39.209229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:39.209229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:39.209229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:39.209229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:39.209229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:39.209229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:39.209229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:39.209229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:39.209229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:39.209229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:39.209229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:39.209229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:39.209229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:39.210721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:39.210721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:39.210721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:39.210721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:39.210721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:39.210721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:39.210721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:39.210721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:39.210721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:39.210721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:39.210721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:39.210721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:39.210721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:39.210721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:39.210721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:39.210721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:39.210721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:39.212459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:39.212459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:39.212459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:39.212459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:39.212459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 238/458] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:39.221527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606f629-646c-4afd-aaf6-27eb9905fc5d None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:39.409535 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:39.459389 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606f629-646c-4afd-aaf6-27eb9905fc5d None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:39.516740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c434af-ba13-4403-84a0-c162fa21628b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:39.517421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c434af-ba13-4403-84a0-c162fa21628b None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:39.517703 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c434af-ba13-4403-84a0-c162fa21628b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:39.517979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c434af-ba13-4403-84a0-c162fa21628b None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:39.518729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c434af-ba13-4403-84a0-c162fa21628b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:39.519065 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c434af-ba13-4403-84a0-c162fa21628b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:39.520458 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c434af-ba13-4403-84a0-c162fa21628b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:39.525580 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547aef7b-21be-4861-b428-a202ddf4d2a1 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:39.526079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547aef7b-21be-4861-b428-a202ddf4d2a1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:39.526555 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547aef7b-21be-4861-b428-a202ddf4d2a1 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:39.529103 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 239/459] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:49:39.536546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2aa774-fe1e-438f-8e6b-5f150af85242 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:39.536809 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2aa774-fe1e-438f-8e6b-5f150af85242 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:39.537099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2aa774-fe1e-438f-8e6b-5f150af85242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:39.545134 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 221/460] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:39.557321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c434af-ba13-4403-84a0-c162fa21628b None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:39.571992 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-1c2aa774-fe1e-438f-8e6b-5f150af85242 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:39.573222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1c2aa774-fe1e-438f-8e6b-5f150af85242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:39.710728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 240/461] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:39.752282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2aa774-fe1e-438f-8e6b-5f150af85242 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:39.777422 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c434af-ba13-4403-84a0-c162fa21628b None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:39.862443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:39.885054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b60cf-b3de-472e-b8db-c257ba8d1f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:39.886243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b60cf-b3de-472e-b8db-c257ba8d1f9b None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:39.886731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b60cf-b3de-472e-b8db-c257ba8d1f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:39.887268 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b60cf-b3de-472e-b8db-c257ba8d1f9b None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:39.888674 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b60cf-b3de-472e-b8db-c257ba8d1f9b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:39.893692 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b60cf-b3de-472e-b8db-c257ba8d1f9b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:39.912367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:40.012021 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2aa774-fe1e-438f-8e6b-5f150af85242 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:40.051206 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 222/462] 10.222.0.27 () {60 vars in 1132 bytes} [Sat Jul 16 14:49:39 2022] POST /identity/v3/users => generated 246 bytes in 495 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 14:49:40.055818 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11155a3-0484-49a0-911a-559f1356bffd None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:40.055818 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11155a3-0484-49a0-911a-559f1356bffd None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:40.055818 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11155a3-0484-49a0-911a-559f1356bffd None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:40.062666 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 223/463] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:40.065898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00780978-9789-45b5-90fb-997d5b582190 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:40.066356 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00780978-9789-45b5-90fb-997d5b582190 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:40.068990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00780978-9789-45b5-90fb-997d5b582190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:40.112804 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-00780978-9789-45b5-90fb-997d5b582190 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:40.115990 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1122760-0aee-4b91-9704-c8774e0e830a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:40.118918 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-00780978-9789-45b5-90fb-997d5b582190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:40.119374 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1122760-0aee-4b91-9704-c8774e0e830a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:40.120083 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1122760-0aee-4b91-9704-c8774e0e830a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:40.120784 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1122760-0aee-4b91-9704-c8774e0e830a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:40.141867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1122760-0aee-4b91-9704-c8774e0e830a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:40.144367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1122760-0aee-4b91-9704-c8774e0e830a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:40.181351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-c1122760-0aee-4b91-9704-c8774e0e830a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:40.181351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:40.181351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:40.181351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:40.181351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:40.181351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:40.181351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:40.181351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:40.181351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:40.181351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:40.181351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:40.181351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:40.181351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:40.181351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:40.181351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:40.183008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:40.183008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:40.183008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:40.183008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:40.183008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:40.183008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:40.183008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:40.183008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:40.183008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:40.183008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:40.183008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:40.183008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:40.183008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:40.183008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:40.183008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:40.183008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:40.183008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:40.185184 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:40.185184 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:40.185184 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:40.185184 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:40.187420 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 241/464] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 435 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:49:40.190440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ade79-df2a-4b51-b802-17871c19ab26 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:40.191044 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ade79-df2a-4b51-b802-17871c19ab26 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:40.191535 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ade79-df2a-4b51-b802-17871c19ab26 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:40.193837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 242/465] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:40.199374 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ade79-df2a-4b51-b802-17871c19ab26 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:40.270394 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 224/466] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 14:49:40.271882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf8894-af96-4088-9cb0-60c67350cbe9 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:40.272228 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf8894-af96-4088-9cb0-60c67350cbe9 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:40.272568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf8894-af96-4088-9cb0-60c67350cbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:40.292855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-81bf8894-af96-4088-9cb0-60c67350cbe9 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:40.294169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-81bf8894-af96-4088-9cb0-60c67350cbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:40.410541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 225/467] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:40.412366 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf8894-af96-4088-9cb0-60c67350cbe9 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:40.431388 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ade79-df2a-4b51-b802-17871c19ab26 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:40.448435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:40.485242 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d73fe4-8c84-4fde-a66c-f702f26650bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:40.485928 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d73fe4-8c84-4fde-a66c-f702f26650bc None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:40.486204 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d73fe4-8c84-4fde-a66c-f702f26650bc None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:40.486483 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d73fe4-8c84-4fde-a66c-f702f26650bc None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:40.487258 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d73fe4-8c84-4fde-a66c-f702f26650bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:40.487566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d73fe4-8c84-4fde-a66c-f702f26650bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:40.488787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:40.489038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d73fe4-8c84-4fde-a66c-f702f26650bc None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:40.516225 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 243/468] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:40.516225 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d73fe4-8c84-4fde-a66c-f702f26650bc None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:40.576633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf8894-af96-4088-9cb0-60c67350cbe9 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:40.617068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2b4c0-f177-4d0f-8c18-6f89d2efb91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:40.618822 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2b4c0-f177-4d0f-8c18-6f89d2efb91b None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:40.619129 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2b4c0-f177-4d0f-8c18-6f89d2efb91b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:40.619398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2b4c0-f177-4d0f-8c18-6f89d2efb91b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:40.630587 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d2b4c0-f177-4d0f-8c18-6f89d2efb91b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:40.632178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d2b4c0-f177-4d0f-8c18-6f89d2efb91b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:40.646923 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d73fe4-8c84-4fde-a66c-f702f26650bc None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:40.654635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-a6d2b4c0-f177-4d0f-8c18-6f89d2efb91b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:40.654635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:40.654635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:40.654635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:40.654635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:40.654635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:40.654635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:40.654635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:40.654635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:40.654635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:40.654635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:40.654635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:40.654635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:40.654635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:40.654635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:40.656333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:40.656333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:40.656333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:40.656333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:40.656333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:40.656333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:40.656333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:40.656333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:40.656333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:40.656333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:40.656333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:40.656333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:40.656333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:40.656333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:40.656333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:40.656333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:40.656333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:40.658776 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:40.658776 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:40.658776 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:40.658776 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:40.658776 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 226/469] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:40.658776 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2b4c0-f177-4d0f-8c18-6f89d2efb91b None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:40.693571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8ef71-573a-4aa7-a086-6ab3cdbd9926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:40.694372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8ef71-573a-4aa7-a086-6ab3cdbd9926 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:40.694647 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8ef71-573a-4aa7-a086-6ab3cdbd9926 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:40.694938 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8ef71-573a-4aa7-a086-6ab3cdbd9926 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:40.707361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8ef71-573a-4aa7-a086-6ab3cdbd9926 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:40.708845 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8ef71-573a-4aa7-a086-6ab3cdbd9926 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:40.730099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-76a8ef71-573a-4aa7-a086-6ab3cdbd9926 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:40.730099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:40.730099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:40.730099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:40.730099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:40.730099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:40.730099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:40.730099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:40.730099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:40.730099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:40.730099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:40.730099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:40.730099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:40.730099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:40.730099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:40.733026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:40.733026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:40.733026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:40.733026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:40.733026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:40.733026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:40.733026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:40.733026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:40.733026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:40.733026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:40.733026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:40.733026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:40.733026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:40.733026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:40.733026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:40.733026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:40.733026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:40.734729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:40.734729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:40.734729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:40.734729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:40.734729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 244/470] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:40.734729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55054d27-c3d9-4375-8d36-43e177ecc3ea None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:40.734729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55054d27-c3d9-4375-8d36-43e177ecc3ea None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:40.734729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55054d27-c3d9-4375-8d36-43e177ecc3ea None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:40.737195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 245/471] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:40.738996 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55054d27-c3d9-4375-8d36-43e177ecc3ea None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:40.780716 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:40.803299 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:40.810276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2b4c0-f177-4d0f-8c18-6f89d2efb91b None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:40.850996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7653288-b0b6-4f7c-94b4-0b6b5f453c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:40.851589 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7653288-b0b6-4f7c-94b4-0b6b5f453c32 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:40.851880 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7653288-b0b6-4f7c-94b4-0b6b5f453c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:40.852145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7653288-b0b6-4f7c-94b4-0b6b5f453c32 None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:40.852877 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7653288-b0b6-4f7c-94b4-0b6b5f453c32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:40.854339 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7653288-b0b6-4f7c-94b4-0b6b5f453c32 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:40.910608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55054d27-c3d9-4375-8d36-43e177ecc3ea None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:40.952871 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479129b-8c82-4365-9c5c-0f399a7590c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:40.953569 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479129b-8c82-4365-9c5c-0f399a7590c9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:40.953856 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479129b-8c82-4365-9c5c-0f399a7590c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:40.954183 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479129b-8c82-4365-9c5c-0f399a7590c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:40.954999 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479129b-8c82-4365-9c5c-0f399a7590c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:40.955312 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479129b-8c82-4365-9c5c-0f399a7590c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:40.956772 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479129b-8c82-4365-9c5c-0f399a7590c9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:40.967983 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 227/472] 10.222.0.27 () {60 vars in 1132 bytes} [Sat Jul 16 14:49:40 2022] POST /identity/v3/users => generated 248 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:49:40.969432 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3eb8f2-294e-407d-87c3-5f03595cb079 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:40.969793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3eb8f2-294e-407d-87c3-5f03595cb079 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:40.970741 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3eb8f2-294e-407d-87c3-5f03595cb079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:40.993052 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 246/473] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:40.995433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479129b-8c82-4365-9c5c-0f399a7590c9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:41.003341 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-ca3eb8f2-294e-407d-87c3-5f03595cb079 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:41.004810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ca3eb8f2-294e-407d-87c3-5f03595cb079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:41.117854 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:41.145225 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479129b-8c82-4365-9c5c-0f399a7590c9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:41.148005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 228/474] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:41.150529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3eb8f2-294e-407d-87c3-5f03595cb079 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:41.211654 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71a1f1-4c76-4ffa-b48f-c6a47f33911c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:41.212613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71a1f1-4c76-4ffa-b48f-c6a47f33911c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:41.212908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71a1f1-4c76-4ffa-b48f-c6a47f33911c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:41.213173 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71a1f1-4c76-4ffa-b48f-c6a47f33911c None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:41.213929 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71a1f1-4c76-4ffa-b48f-c6a47f33911c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:41.214239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71a1f1-4c76-4ffa-b48f-c6a47f33911c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:41.215723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71a1f1-4c76-4ffa-b48f-c6a47f33911c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:41.232587 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 247/475] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:41.233447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71a1f1-4c76-4ffa-b48f-c6a47f33911c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:41.316276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:41.344550 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3eb8f2-294e-407d-87c3-5f03595cb079 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:41.395375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503395d-a270-48c8-bdce-998c7386f4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:41.396078 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503395d-a270-48c8-bdce-998c7386f4a1 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:41.396399 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503395d-a270-48c8-bdce-998c7386f4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:41.396683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503395d-a270-48c8-bdce-998c7386f4a1 None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:41.397452 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503395d-a270-48c8-bdce-998c7386f4a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:41.399928 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503395d-a270-48c8-bdce-998c7386f4a1 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:41.442782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71a1f1-4c76-4ffa-b48f-c6a47f33911c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:41.506573 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a208f0e-0b74-4504-8057-2aad125e7927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:41.507862 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a208f0e-0b74-4504-8057-2aad125e7927 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:41.508505 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a208f0e-0b74-4504-8057-2aad125e7927 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:41.509374 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a208f0e-0b74-4504-8057-2aad125e7927 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:41.510902 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a208f0e-0b74-4504-8057-2aad125e7927 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:41.513605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a208f0e-0b74-4504-8057-2aad125e7927 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:41.528367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 229/476] 10.222.0.27 () {60 vars in 1132 bytes} [Sat Jul 16 14:49:41 2022] POST /identity/v3/users => generated 248 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:49:41.529577 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503395d-a270-48c8-bdce-998c7386f4a1 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:41.580949 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.sql.core [None req-9a208f0e-0b74-4504-8057-2aad125e7927 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 16 14:49:41.580949 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 16 14:49:41.580949 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [parameters: {'id': '0b6a09691e61415a85302d1408132fd8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 16 14:49:41.580949 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80256) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 16 14:49:41.614017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-9a208f0e-0b74-4504-8057-2aad125e7927 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 16 14:49:41.614017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:41.614017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 16 14:49:41.614017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 16 14:49:41.614017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 16 14:49:41.614017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 16 14:49:41.614017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 16 14:49:41.614017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application result = self._query(query) Jul 16 14:49:41.614017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 16 14:49:41.614017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application conn.query(q) Jul 16 14:49:41.614017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 16 14:49:41.614017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 16 14:49:41.614017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 16 14:49:41.614017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application result.read() Jul 16 14:49:41.614017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 16 14:49:41.619968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 16 14:49:41.619968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 16 14:49:41.619968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 16 14:49:41.619968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 16 14:49:41.619968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 16 14:49:41.619968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 16 14:49:41.619968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 16 14:49:41.619968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 16 14:49:41.619968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:41.619968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 16 14:49:41.619968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:41.619968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:41.619968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 16 14:49:41.619968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 16 14:49:41.619968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 16 14:49:41.619968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return ref.to_dict() Jul 16 14:49:41.619968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 16 14:49:41.619968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application next(self.gen) Jul 16 14:49:41.621566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 16 14:49:41.621566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application yield resource Jul 16 14:49:41.621566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 16 14:49:41.621566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application next(self.gen) Jul 16 14:49:41.621566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 16 14:49:41.621566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self.session.rollback() Jul 16 14:49:41.621566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 14:49:41.621566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self.force_reraise() Jul 16 14:49:41.621566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 14:49:41.621566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise self.value Jul 16 14:49:41.621566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 16 14:49:41.621566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 16 14:49:41.621566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 16 14:49:41.621566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application session.commit() Jul 16 14:49:41.621566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 16 14:49:41.621566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 16 14:49:41.621566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 16 14:49:41.624132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._prepare_impl() Jul 16 14:49:41.624132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 16 14:49:41.624132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self.session.flush() Jul 16 14:49:41.624132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 16 14:49:41.624132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._flush(objects) Jul 16 14:49:41.624132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 16 14:49:41.624132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 16 14:49:41.624132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 16 14:49:41.624132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application compat.raise_( Jul 16 14:49:41.624132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 16 14:49:41.624132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception Jul 16 14:49:41.624132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 16 14:49:41.624132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application flush_context.execute() Jul 16 14:49:41.624132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 16 14:49:41.624132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rec.execute(self) Jul 16 14:49:41.624132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 16 14:49:41.624132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 16 14:49:41.631253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 16 14:49:41.631253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 16 14:49:41.631253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 16 14:49:41.631253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 16 14:49:41.631253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 16 14:49:41.631253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 16 14:49:41.631253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 16 14:49:41.631253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 16 14:49:41.631253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 16 14:49:41.631253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 16 14:49:41.631253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 16 14:49:41.631253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 16 14:49:41.631253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 16 14:49:41.631253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 16 14:49:41.631253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 16 14:49:41.631253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception Jul 16 14:49:41.643550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 16 14:49:41.643550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 16 14:49:41.643550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 16 14:49:41.643550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 16 14:49:41.643550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 16 14:49:41.643550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application result = self._query(query) Jul 16 14:49:41.643550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 16 14:49:41.643550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application conn.query(q) Jul 16 14:49:41.643550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 16 14:49:41.643550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 16 14:49:41.643550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 16 14:49:41.643550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application result.read() Jul 16 14:49:41.643550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 16 14:49:41.643550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 16 14:49:41.643550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 16 14:49:41.643550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 16 14:49:41.646622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 16 14:49:41.646622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 16 14:49:41.646622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 16 14:49:41.646622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 16 14:49:41.646622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 16 14:49:41.646622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 16 14:49:41.646622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [parameters: {'id': '0b6a09691e61415a85302d1408132fd8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 16 14:49:41.646622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 16 14:49:41.646622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:41.646622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:49:41.646622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:41.646622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:41.646622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:41.646622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:41.646622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:41.646622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:41.649882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:41.649882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:41.649882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:41.649882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:41.649882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:41.649882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:41.649882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 16 14:49:41.649882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 16 14:49:41.649882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:41.649882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:41.649882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 16 14:49:41.649882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 16 14:49:41.649882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 16 14:49:41.649882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 16 14:49:41.649882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 16 14:49:41.649882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:41.660377 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 248/477] 10.222.0.27 () {60 vars in 1131 bytes} [Sat Jul 16 14:49:41 2022] POST /identity/v3/roles => generated 144 bytes in 384 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:41.660377 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a208f0e-0b74-4504-8057-2aad125e7927 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:41.660377 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:41.763010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503395d-a270-48c8-bdce-998c7386f4a1 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:41.829845 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c90bfd-dd9f-4035-98a4-bdb42ccfcbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:41.831122 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c90bfd-dd9f-4035-98a4-bdb42ccfcbb3 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:41.831574 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c90bfd-dd9f-4035-98a4-bdb42ccfcbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:41.832034 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c90bfd-dd9f-4035-98a4-bdb42ccfcbb3 None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:41.833282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c90bfd-dd9f-4035-98a4-bdb42ccfcbb3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:41.836704 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c90bfd-dd9f-4035-98a4-bdb42ccfcbb3 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:41.855894 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:41.875793 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a208f0e-0b74-4504-8057-2aad125e7927 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:41.935759 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58a886-e5f6-42b0-9033-8d4b1b1c2f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:41.936582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58a886-e5f6-42b0-9033-8d4b1b1c2f0c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:41.936859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58a886-e5f6-42b0-9033-8d4b1b1c2f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:41.937130 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58a886-e5f6-42b0-9033-8d4b1b1c2f0c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:41.952741 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb58a886-e5f6-42b0-9033-8d4b1b1c2f0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:41.961553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-fb58a886-e5f6-42b0-9033-8d4b1b1c2f0c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:49:41.961553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:41.961553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:41.961553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:41.961553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:41.961553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:41.961553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:41.961553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:41.961553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:41.961553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:41.961553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:41.961553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:41.961553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:41.961553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:41.961553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:41.961553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:41.963516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:41.963516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:41.963516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:41.963516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:41.963516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:41.963516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:41.963516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:41.963516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:41.963516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:41.963516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:41.963516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:41.963516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:41.963516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:41.963516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:41.963516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:41.963516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:41.964955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:41.964955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:41.964955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:41.964955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:41.964955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:41.964955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:41.964955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:41.964955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:41.964955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:41.964955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:41.964955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 16 14:49:41.964955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:41.964955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 249/478] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:49:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:41.969342 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58a886-e5f6-42b0-9033-8d4b1b1c2f0c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:41.994874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 230/479] 10.222.0.27 () {60 vars in 1132 bytes} [Sat Jul 16 14:49:41 2022] POST /identity/v3/users => generated 249 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:49:42.060374 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:42.108542 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58a886-e5f6-42b0-9033-8d4b1b1c2f0c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:42.167722 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc81dd0-a3c0-4341-9ab0-5bce820e7da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:42.168309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc81dd0-a3c0-4341-9ab0-5bce820e7da6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:42.168590 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc81dd0-a3c0-4341-9ab0-5bce820e7da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:42.168873 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc81dd0-a3c0-4341-9ab0-5bce820e7da6 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:42.169662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc81dd0-a3c0-4341-9ab0-5bce820e7da6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:42.169970 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc81dd0-a3c0-4341-9ab0-5bce820e7da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:42.171492 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc81dd0-a3c0-4341-9ab0-5bce820e7da6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:42.185896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 250/480] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:49:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:49:43.704964 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3085606-2ee4-498b-9c1c-4f62836816fe None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:43.705659 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3085606-2ee4-498b-9c1c-4f62836816fe None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:43.705659 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3085606-2ee4-498b-9c1c-4f62836816fe None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:43.707324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 231/481] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:43.714654 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53b323-de04-41ad-9ba6-a151b14de189 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:43.715428 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53b323-de04-41ad-9ba6-a151b14de189 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:43.715428 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53b323-de04-41ad-9ba6-a151b14de189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:43.740047 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ed53b323-de04-41ad-9ba6-a151b14de189 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:43.741282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ed53b323-de04-41ad-9ba6-a151b14de189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:43.839222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 251/482] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:43.871068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3085606-2ee4-498b-9c1c-4f62836816fe None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:44.012277 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3085606-2ee4-498b-9c1c-4f62836816fe None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:44.033009 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99667a-5656-493f-8239-8086ccfef678 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:44.033450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99667a-5656-493f-8239-8086ccfef678 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:44.033843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99667a-5656-493f-8239-8086ccfef678 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:44.035652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 252/483] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:44.043519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abe3c6-b3c4-434b-b4fa-4974362964f0 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:44.043929 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abe3c6-b3c4-434b-b4fa-4974362964f0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:44.044304 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abe3c6-b3c4-434b-b4fa-4974362964f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:44.052138 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203720a-95cc-47ca-8807-404fb470330d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:44.053209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203720a-95cc-47ca-8807-404fb470330d None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:44.053768 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203720a-95cc-47ca-8807-404fb470330d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:44.054183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203720a-95cc-47ca-8807-404fb470330d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:44.067009 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203720a-95cc-47ca-8807-404fb470330d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:44.076037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-8203720a-95cc-47ca-8807-404fb470330d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:49:44.076037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:44.076037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:44.076037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:44.076037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:44.076037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:44.076037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:44.076037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:44.076037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:44.076037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:44.076037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:44.076037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:44.076037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:44.076037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:44.076037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:44.076037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:44.077236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:44.077236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:44.077236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:44.077236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:44.077236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:44.077236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:44.077236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:44.077236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:44.077236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:44.077236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:44.077236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:44.077236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:44.077236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:44.077236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:44.077236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:44.077236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:44.078214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:44.078214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:44.078214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:44.078214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:44.078214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:44.078214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:44.078214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:44.078214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:44.078214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:44.078214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:44.078214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:49:44.078214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:44.078976 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-72abe3c6-b3c4-434b-b4fa-4974362964f0 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:44.080025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-72abe3c6-b3c4-434b-b4fa-4974362964f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:44.080953 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 232/484] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:49:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:44.087977 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203720a-95cc-47ca-8807-404fb470330d None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:44.190839 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:44.213881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:44.233622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 253/485] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:44.258288 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72abe3c6-b3c4-434b-b4fa-4974362964f0 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:44.365542 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203720a-95cc-47ca-8807-404fb470330d None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:44.444380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26690c-255f-4839-824c-80753c916ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:44.446823 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26690c-255f-4839-824c-80753c916ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:44.447346 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26690c-255f-4839-824c-80753c916ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:44.447826 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26690c-255f-4839-824c-80753c916ab8 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:44.449269 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26690c-255f-4839-824c-80753c916ab8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:44.449739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26690c-255f-4839-824c-80753c916ab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:44.455960 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26690c-255f-4839-824c-80753c916ab8 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:44.460280 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72abe3c6-b3c4-434b-b4fa-4974362964f0 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:44.478453 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 233/486] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:49:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:44.482009 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26690c-255f-4839-824c-80753c916ab8 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:44.576264 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbaa656-7e75-489f-b099-9612564abcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:44.577089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbaa656-7e75-489f-b099-9612564abcff None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:44.577373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbaa656-7e75-489f-b099-9612564abcff None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:44.577652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbaa656-7e75-489f-b099-9612564abcff None admin] PATH_INFO: `/v3/roles/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:44.599963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbaa656-7e75-489f-b099-9612564abcff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:44.616677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-fbbaa656-7e75-489f-b099-9612564abcff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:49:44.616677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:44.616677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:44.616677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:44.616677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:44.616677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:44.616677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:44.616677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:44.616677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:44.616677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:44.616677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:44.616677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:44.616677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:44.616677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:44.616677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:44.616677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:44.619087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:44.619087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:44.619087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:44.619087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:44.619087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:44.619087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:44.619087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:44.619087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:44.619087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:44.619087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:44.619087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:44.619087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:44.619087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:44.619087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:44.619087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:44.619087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:44.620570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:44.620570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:44.620570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:44.620570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:44.620570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:44.620570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:44.620570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:44.620570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:44.620570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:44.620570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:44.620570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:49:44.620570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:44.630924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 254/487] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:49:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:49:44.630924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accb38c-9498-41bf-8fa0-a61ce1152388 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:44.630924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accb38c-9498-41bf-8fa0-a61ce1152388 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:44.630924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accb38c-9498-41bf-8fa0-a61ce1152388 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:44.630924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 255/488] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:44.630924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accb38c-9498-41bf-8fa0-a61ce1152388 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:44.650965 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26690c-255f-4839-824c-80753c916ab8 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:44.696171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:44.705358 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:44.732115 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b3932-e22f-406f-b53c-6e0dc584f2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:44.735367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b3932-e22f-406f-b53c-6e0dc584f2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:44.736000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b3932-e22f-406f-b53c-6e0dc584f2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:44.736532 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b3932-e22f-406f-b53c-6e0dc584f2a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:44.764272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b3932-e22f-406f-b53c-6e0dc584f2a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:44.768180 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b3932-e22f-406f-b53c-6e0dc584f2a6 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:44.792167 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accb38c-9498-41bf-8fa0-a61ce1152388 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:44.808350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-1d5b3932-e22f-406f-b53c-6e0dc584f2a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:44.808350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:44.808350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:44.808350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:44.808350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:44.808350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:44.808350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:44.808350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:44.808350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:44.808350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:44.808350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:44.808350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:44.808350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:44.808350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:44.808350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:44.810209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:44.810209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:44.810209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:44.810209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:44.810209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:44.810209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:44.810209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:44.810209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:44.810209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:44.810209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:44.810209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:44.810209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:44.810209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:44.810209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:44.810209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:44.810209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:44.810209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:44.812237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:44.812237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:44.812237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:44.812237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:44.818934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 234/489] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:44.818934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5982114-b815-4038-a34c-1905f4ccb832 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:44.818934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5982114-b815-4038-a34c-1905f4ccb832 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:44.818934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5982114-b815-4038-a34c-1905f4ccb832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:44.840229 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-e5982114-b815-4038-a34c-1905f4ccb832 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:44.842063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e5982114-b815-4038-a34c-1905f4ccb832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:44.874970 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06e52b-e7be-4f10-9081-9f3b3574c08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:44.876519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06e52b-e7be-4f10-9081-9f3b3574c08a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:44.877241 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06e52b-e7be-4f10-9081-9f3b3574c08a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:44.877747 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06e52b-e7be-4f10-9081-9f3b3574c08a None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:44.879433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06e52b-e7be-4f10-9081-9f3b3574c08a None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:44.880285 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06e52b-e7be-4f10-9081-9f3b3574c08a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:44.883408 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06e52b-e7be-4f10-9081-9f3b3574c08a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:44.901694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 256/490] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:49:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:44.903600 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06e52b-e7be-4f10-9081-9f3b3574c08a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:44.978998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 235/491] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:44.978998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5982114-b815-4038-a34c-1905f4ccb832 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:45.114496 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06e52b-e7be-4f10-9081-9f3b3574c08a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:45.116861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:45.122643 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5982114-b815-4038-a34c-1905f4ccb832 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:45.126456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:45.164117 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00578272-091d-4581-968a-97903180f598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:45.165234 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00578272-091d-4581-968a-97903180f598 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:45.165742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00578272-091d-4581-968a-97903180f598 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:45.166282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00578272-091d-4581-968a-97903180f598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:45.170447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6868e504-3590-4634-acd7-ebcc718498d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:45.171646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868e504-3590-4634-acd7-ebcc718498d9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:45.172135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868e504-3590-4634-acd7-ebcc718498d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:45.172489 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6868e504-3590-4634-acd7-ebcc718498d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:45.173859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6868e504-3590-4634-acd7-ebcc718498d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:45.174255 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6868e504-3590-4634-acd7-ebcc718498d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:45.178210 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6868e504-3590-4634-acd7-ebcc718498d9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:45.179090 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00578272-091d-4581-968a-97903180f598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:45.181929 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00578272-091d-4581-968a-97903180f598 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:45.201219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-00578272-091d-4581-968a-97903180f598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:45.201219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:45.201219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:45.201219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:45.201219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:45.201219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:45.201219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:45.201219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:45.201219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:45.201219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:45.201219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:45.201219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:45.201219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:45.201219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:45.201219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:45.202820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:45.202820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:45.202820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:45.202820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:45.202820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:45.202820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:45.202820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:45.202820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:45.202820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:45.202820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:45.202820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:45.202820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:45.202820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:45.202820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:45.202820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:45.202820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:45.202820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:45.204578 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:45.204578 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:45.204578 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:45.204578 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:45.207069 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 236/492] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:45.207543 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 257/493] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:45.210319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00578272-091d-4581-968a-97903180f598 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:45.210590 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fd428-f084-4704-ab25-409a5132cba1 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:45.210590 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fd428-f084-4704-ab25-409a5132cba1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:45.210590 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fd428-f084-4704-ab25-409a5132cba1 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:45.214993 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 258/494] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:45.215732 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766fd428-f084-4704-ab25-409a5132cba1 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:45.340126 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00578272-091d-4581-968a-97903180f598 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:45.343150 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766fd428-f084-4704-ab25-409a5132cba1 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:45.378053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9cd83-f26d-447e-88a6-0a988e3ca3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:45.378743 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9cd83-f26d-447e-88a6-0a988e3ca3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:45.379139 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9cd83-f26d-447e-88a6-0a988e3ca3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:45.379329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9cd83-f26d-447e-88a6-0a988e3ca3d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:45.385857 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46644be7-1596-498f-8121-1e3a66ac9666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:45.386515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46644be7-1596-498f-8121-1e3a66ac9666 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:45.386800 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46644be7-1596-498f-8121-1e3a66ac9666 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:45.387096 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46644be7-1596-498f-8121-1e3a66ac9666 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:45.387830 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46644be7-1596-498f-8121-1e3a66ac9666 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:45.388128 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46644be7-1596-498f-8121-1e3a66ac9666 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:45.389509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46644be7-1596-498f-8121-1e3a66ac9666 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:45.391070 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9cd83-f26d-447e-88a6-0a988e3ca3d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:45.399596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-46b9cd83-f26d-447e-88a6-0a988e3ca3d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:49:45.399596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:45.399596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:45.399596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:45.399596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:45.399596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:45.399596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:45.399596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:45.399596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:45.399596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:45.399596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:45.399596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:45.399596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:45.399596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:45.399596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:45.399596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:45.400777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:45.400777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:45.400777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:45.400777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:45.400777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:45.400777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:45.400777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:45.400777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:45.400777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:45.400777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:45.400777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:45.400777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:45.400777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:45.400777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:45.400777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:45.400777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:45.401581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:45.401581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:45.401581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:45.401581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:45.401581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:45.401581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:45.401581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:45.401581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:45.401581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:45.401581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:45.401581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:49:45.401581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:45.401581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 237/495] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:49:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:45.401581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9cd83-f26d-447e-88a6-0a988e3ca3d2 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:45.402319 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 259/496] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:45.403484 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd94b44-4a3d-482b-b52f-c1c2fc1e35ea None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:45.403771 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd94b44-4a3d-482b-b52f-c1c2fc1e35ea None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:45.404044 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd94b44-4a3d-482b-b52f-c1c2fc1e35ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:45.422290 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:45.432495 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-7fd94b44-4a3d-482b-b52f-c1c2fc1e35ea None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:45.433694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7fd94b44-4a3d-482b-b52f-c1c2fc1e35ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:45.455576 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:45.508317 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 260/497] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:45.509803 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f59c8b-476b-46f2-814b-ca7830c7843c None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:45.510094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f59c8b-476b-46f2-814b-ca7830c7843c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:45.510369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f59c8b-476b-46f2-814b-ca7830c7843c None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:45.515062 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 261/498] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:45.515062 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f59c8b-476b-46f2-814b-ca7830c7843c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:45.523457 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9cd83-f26d-447e-88a6-0a988e3ca3d2 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:45.570361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bdc07c-778f-4ba0-ba50-81b7cbfddc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:45.571198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdc07c-778f-4ba0-ba50-81b7cbfddc60 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:45.571502 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdc07c-778f-4ba0-ba50-81b7cbfddc60 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:45.571747 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdc07c-778f-4ba0-ba50-81b7cbfddc60 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:45.579913 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bdc07c-778f-4ba0-ba50-81b7cbfddc60 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:45.581396 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bdc07c-778f-4ba0-ba50-81b7cbfddc60 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:45.599526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-84bdc07c-778f-4ba0-ba50-81b7cbfddc60 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 16 14:49:45.599526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:45.599526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:45.599526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:45.599526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:45.599526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:45.599526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:45.599526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:45.599526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:45.599526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:45.599526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:45.599526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:45.599526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:49:45.599526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:49:45.599526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:49:45.599526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:49:45.600493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:45.600493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:45.600493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:49:45.600493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:49:45.600493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:49:45.600493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:49:45.600493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:45.600493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:45.600493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:45.600493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:45.600493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:45.600493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:45.600493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:45.600493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:45.600493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:45.600493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:45.600493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:45.600493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:45.601883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:45.601883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:45.601883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:49:45.601883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:49:45.601883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:49:45.601883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:49:45.601883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:49:45.601883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:49:45.601883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 16 14:49:45.601883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:45.601883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 238/499] 10.222.0.27 () {56 vars in 1093 bytes} [Sat Jul 16 14:49:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:45.602629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bdc07c-778f-4ba0-ba50-81b7cbfddc60 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:45.649081 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f59c8b-476b-46f2-814b-ca7830c7843c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:45.685714 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84ae76-01bf-4de7-bea5-f54c5734bacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:45.686510 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84ae76-01bf-4de7-bea5-f54c5734bacd None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:45.686793 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84ae76-01bf-4de7-bea5-f54c5734bacd None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:45.687095 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84ae76-01bf-4de7-bea5-f54c5734bacd None admin] PATH_INFO: `/v3/users/glance` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:45.688175 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:45.695082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a84ae76-01bf-4de7-bea5-f54c5734bacd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:45.696600 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a84ae76-01bf-4de7-bea5-f54c5734bacd None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:45.713870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-3a84ae76-01bf-4de7-bea5-f54c5734bacd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 16 14:49:45.713870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:45.713870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:45.713870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:45.713870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:45.713870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:45.713870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:45.713870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:45.713870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:45.713870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:45.713870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:45.713870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:45.713870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:49:45.713870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:49:45.713870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:49:45.713870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:49:45.715962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:45.715962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:45.715962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:49:45.715962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:49:45.715962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:49:45.715962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:49:45.715962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:45.715962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:45.715962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:45.715962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:45.715962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:45.715962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:45.715962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:45.715962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:45.715962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:45.715962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:45.715962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:45.715962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:45.716836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:45.716836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:45.716836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:49:45.716836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:49:45.716836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:49:45.716836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:49:45.716836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:49:45.716836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:49:45.716836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 16 14:49:45.716836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:45.716836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 262/500] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:45.716836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8f564-408c-4417-86cd-4dbea653f044 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:45.716836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8f564-408c-4417-86cd-4dbea653f044 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:45.717509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8f564-408c-4417-86cd-4dbea653f044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:45.732729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bdc07c-778f-4ba0-ba50-81b7cbfddc60 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:45.734514 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ebd8f564-408c-4417-86cd-4dbea653f044 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:45.735766 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ebd8f564-408c-4417-86cd-4dbea653f044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:45.748187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:45.774385 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f75af-ba3f-4b83-aa69-a5c83693155e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:45.775243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f75af-ba3f-4b83-aa69-a5c83693155e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:45.775378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f75af-ba3f-4b83-aa69-a5c83693155e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:45.775640 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f75af-ba3f-4b83-aa69-a5c83693155e None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:45.776649 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f75af-ba3f-4b83-aa69-a5c83693155e None admin] RBAC: Adding query filter params (name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:45.776757 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f75af-ba3f-4b83-aa69-a5c83693155e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:45.778139 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f75af-ba3f-4b83-aa69-a5c83693155e None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:45.786982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 239/501] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:49:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:45.787901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f75af-ba3f-4b83-aa69-a5c83693155e None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:45.818964 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 263/502] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:45.819879 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8f564-408c-4417-86cd-4dbea653f044 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:45.930805 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f75af-ba3f-4b83-aa69-a5c83693155e None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:45.954468 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8f564-408c-4417-86cd-4dbea653f044 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:45.963860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:45.968881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a49d4-84b2-4e23-9b81-4be434fcce5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:45.969675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a49d4-84b2-4e23-9b81-4be434fcce5f None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:45.969949 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a49d4-84b2-4e23-9b81-4be434fcce5f None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:45.970216 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a49d4-84b2-4e23-9b81-4be434fcce5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:45.986945 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a49d4-84b2-4e23-9b81-4be434fcce5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:45.995203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-7d2a49d4-84b2-4e23-9b81-4be434fcce5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:49:45.995203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:45.995203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:45.995203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:45.995203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:45.995203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:45.995203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:45.995203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:45.995203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:45.995203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:45.995203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:45.995203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:45.995203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:45.995203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:45.995203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:45.995203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:45.996278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:45.996278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:45.996278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:45.996278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:45.996278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:45.996278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:45.996278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:45.996278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:45.996278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:45.996278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:45.996278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:45.996278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:45.996278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:45.996278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:45.996278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:45.996278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:45.997133 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:45.997133 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:45.997133 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:45.997133 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:45.997133 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:45.997133 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:45.997133 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:45.997133 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:45.997133 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:45.997133 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:45.997133 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:49:45.997133 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:45.997133 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 240/503] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:49:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:45.998054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a49d4-84b2-4e23-9b81-4be434fcce5f None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:46.007927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e188c-9e45-4aad-b25f-b13926f7acc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:46.009058 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e188c-9e45-4aad-b25f-b13926f7acc8 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:46.009340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e188c-9e45-4aad-b25f-b13926f7acc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:46.009620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e188c-9e45-4aad-b25f-b13926f7acc8 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:46.010485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3e188c-9e45-4aad-b25f-b13926f7acc8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:46.011086 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3e188c-9e45-4aad-b25f-b13926f7acc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:46.012280 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3e188c-9e45-4aad-b25f-b13926f7acc8 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:46.051614 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 264/504] 10.222.0.27 () {56 vars in 1133 bytes} [Sat Jul 16 14:49:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:46.054343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e188c-9e45-4aad-b25f-b13926f7acc8 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:46.059938 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:46.116435 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a49d4-84b2-4e23-9b81-4be434fcce5f None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:46.156193 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327cdd0-8fd2-4ef9-9765-aec58d5e89e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:46.156795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327cdd0-8fd2-4ef9-9765-aec58d5e89e8 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:46.157081 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327cdd0-8fd2-4ef9-9765-aec58d5e89e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:46.157361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327cdd0-8fd2-4ef9-9765-aec58d5e89e8 None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:46.158216 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327cdd0-8fd2-4ef9-9765-aec58d5e89e8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:46.158525 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327cdd0-8fd2-4ef9-9765-aec58d5e89e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:46.160015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327cdd0-8fd2-4ef9-9765-aec58d5e89e8 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:46.169973 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e188c-9e45-4aad-b25f-b13926f7acc8 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:46.201758 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 241/505] 10.222.0.27 () {56 vars in 1137 bytes} [Sat Jul 16 14:49:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:46.202614 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327cdd0-8fd2-4ef9-9765-aec58d5e89e8 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:46.208668 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e8459-10e6-4d7b-af02-d34afeb3d878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:46.209472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e8459-10e6-4d7b-af02-d34afeb3d878 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:46.209750 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e8459-10e6-4d7b-af02-d34afeb3d878 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:46.210018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e8459-10e6-4d7b-af02-d34afeb3d878 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:46.225162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e8459-10e6-4d7b-af02-d34afeb3d878 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:46.227891 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e8459-10e6-4d7b-af02-d34afeb3d878 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:46.259769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-fa8e8459-10e6-4d7b-af02-d34afeb3d878 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:46.259769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:46.259769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:46.259769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:46.259769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:46.259769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:46.259769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:46.259769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:46.259769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:46.259769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:46.259769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:46.259769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:46.259769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:46.259769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:46.259769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:46.260685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:46.260685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:46.260685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:46.260685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:46.260685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:46.260685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:46.260685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:46.260685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:46.260685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:46.260685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:46.260685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:46.260685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:46.260685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:46.260685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:46.260685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:46.260685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:46.260685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:46.261567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:46.261567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:46.261567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:46.261567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:46.262004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 265/506] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:46.263343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e8459-10e6-4d7b-af02-d34afeb3d878 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:46.267483 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:46.322099 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327cdd0-8fd2-4ef9-9765-aec58d5e89e8 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:46.331236 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:46.358198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34df38-7473-4423-a20d-4a7158d723e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:46.359159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34df38-7473-4423-a20d-4a7158d723e2 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:46.359449 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34df38-7473-4423-a20d-4a7158d723e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:46.359848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34df38-7473-4423-a20d-4a7158d723e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:46.372200 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34df38-7473-4423-a20d-4a7158d723e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:46.373681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34df38-7473-4423-a20d-4a7158d723e2 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:46.378328 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e8459-10e6-4d7b-af02-d34afeb3d878 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:46.392037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-0a34df38-7473-4423-a20d-4a7158d723e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:46.392037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:46.392037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:46.392037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:46.392037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:46.392037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:46.392037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:46.392037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:46.392037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:46.392037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:46.392037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:46.392037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:46.392037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:46.392037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:46.392037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:46.393378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:46.393378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:46.393378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:46.393378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:46.393378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:46.393378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:46.393378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:46.393378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:46.393378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:46.393378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:46.393378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:46.393378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:46.393378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:46.393378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:46.393378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:46.393378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:46.393378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:46.394727 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:46.394727 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:46.394727 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:46.394727 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:46.394727 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 242/507] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:46.394727 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34df38-7473-4423-a20d-4a7158d723e2 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:46.418713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269835e-dad7-41f2-a0a2-369eedc3c76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:46.419358 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269835e-dad7-41f2-a0a2-369eedc3c76b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:46.419642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269835e-dad7-41f2-a0a2-369eedc3c76b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:46.419945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269835e-dad7-41f2-a0a2-369eedc3c76b None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:46.420722 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b269835e-dad7-41f2-a0a2-369eedc3c76b None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:46.421031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b269835e-dad7-41f2-a0a2-369eedc3c76b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:46.422473 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b269835e-dad7-41f2-a0a2-369eedc3c76b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:46.431966 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 266/508] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:49:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:46.432819 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269835e-dad7-41f2-a0a2-369eedc3c76b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:46.512711 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34df38-7473-4423-a20d-4a7158d723e2 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:46.514454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:46.546545 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b269835e-dad7-41f2-a0a2-369eedc3c76b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:46.550786 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf9eb1-0a83-4793-9494-fad3207a1a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:46.551460 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf9eb1-0a83-4793-9494-fad3207a1a4b None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:46.551743 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf9eb1-0a83-4793-9494-fad3207a1a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:46.552021 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf9eb1-0a83-4793-9494-fad3207a1a4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:46.565333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:46.587238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252e050-27be-4a9a-998a-47ecff7342bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:46.587901 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252e050-27be-4a9a-998a-47ecff7342bb None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:46.588180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252e050-27be-4a9a-998a-47ecff7342bb None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:46.588458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252e050-27be-4a9a-998a-47ecff7342bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:46.635131 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf9eb1-0a83-4793-9494-fad3207a1a4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:46.636633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf9eb1-0a83-4793-9494-fad3207a1a4b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:46.677288 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252e050-27be-4a9a-998a-47ecff7342bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:46.679132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252e050-27be-4a9a-998a-47ecff7342bb None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:46.752731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 243/509] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:49:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:49:46.754143 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf9eb1-0a83-4793-9494-fad3207a1a4b None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:46.756730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:46.802207 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 267/510] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:49:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:49:46.803184 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252e050-27be-4a9a-998a-47ecff7342bb None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:46.864749 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:46.920718 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf9eb1-0a83-4793-9494-fad3207a1a4b None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:46.959214 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252e050-27be-4a9a-998a-47ecff7342bb None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:46.961505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deaee05-04a5-4b01-a89f-956bb6c474d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:46.962458 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaee05-04a5-4b01-a89f-956bb6c474d0 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:46.962742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaee05-04a5-4b01-a89f-956bb6c474d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:46.963061 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deaee05-04a5-4b01-a89f-956bb6c474d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:46.963792 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deaee05-04a5-4b01-a89f-956bb6c474d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:46.964122 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deaee05-04a5-4b01-a89f-956bb6c474d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:46.965530 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deaee05-04a5-4b01-a89f-956bb6c474d0 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:46.980026 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 244/511] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:46.980946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deaee05-04a5-4b01-a89f-956bb6c474d0 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:47.002691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c4ced2-b391-4c29-a10c-22ac46375865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:47.003414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4ced2-b391-4c29-a10c-22ac46375865 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:47.003689 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4ced2-b391-4c29-a10c-22ac46375865 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:47.003958 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4ced2-b391-4c29-a10c-22ac46375865 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:47.004686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c4ced2-b391-4c29-a10c-22ac46375865 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:47.004990 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c4ced2-b391-4c29-a10c-22ac46375865 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:47.006406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c4ced2-b391-4c29-a10c-22ac46375865 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:47.019633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 268/512] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:47.020601 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c4ced2-b391-4c29-a10c-22ac46375865 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:47.060712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:47.103310 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deaee05-04a5-4b01-a89f-956bb6c474d0 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:47.123752 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:47.141307 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf5f43-1106-432d-9909-6ff443e48dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:47.142382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf5f43-1106-432d-9909-6ff443e48dfa None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:47.142788 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf5f43-1106-432d-9909-6ff443e48dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:47.143177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf5f43-1106-432d-9909-6ff443e48dfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:47.154167 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c4ced2-b391-4c29-a10c-22ac46375865 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:47.154359 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf5f43-1106-432d-9909-6ff443e48dfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:47.155817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf5f43-1106-432d-9909-6ff443e48dfa None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:47.175051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-feaf5f43-1106-432d-9909-6ff443e48dfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:47.175051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:47.175051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:47.175051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:47.175051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:47.175051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:47.175051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:47.175051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:47.175051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:47.175051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:47.175051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:47.175051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:47.175051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:47.175051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:47.175051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:47.175961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:47.175961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:47.175961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:47.175961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:47.175961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:47.175961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:47.175961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:47.175961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:47.175961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:47.175961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:47.175961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:47.175961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:47.175961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:47.175961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:47.175961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:47.175961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:47.175961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:47.176818 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:47.176818 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:47.176818 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:47.176818 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:47.180186 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 245/513] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:47.181124 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf5f43-1106-432d-9909-6ff443e48dfa None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:47.214185 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb210de-4e95-470e-a158-c9e122b74004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:47.215491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb210de-4e95-470e-a158-c9e122b74004 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:47.216001 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb210de-4e95-470e-a158-c9e122b74004 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:47.216499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb210de-4e95-470e-a158-c9e122b74004 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:47.232274 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb210de-4e95-470e-a158-c9e122b74004 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:47.233748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb210de-4e95-470e-a158-c9e122b74004 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:47.262114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-4bb210de-4e95-470e-a158-c9e122b74004 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:47.262114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:47.262114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:47.262114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:47.262114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:47.262114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:47.262114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:47.262114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:47.262114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:47.262114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:47.262114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:47.262114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:47.262114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:47.262114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:47.262114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:47.263104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:47.263104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:47.263104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:47.263104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:47.263104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:47.263104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:47.263104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:47.263104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:47.263104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:47.263104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:47.263104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:47.263104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:47.263104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:47.263104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:47.263104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:47.263104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:47.263104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:47.263938 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:47.263938 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:47.263938 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:47.263938 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:47.264871 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 269/514] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:47.266349 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb210de-4e95-470e-a158-c9e122b74004 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:47.291968 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf5f43-1106-432d-9909-6ff443e48dfa None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:47.314508 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:47.339210 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c669b-8da1-4f89-9052-20d2a247c180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:47.340557 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c669b-8da1-4f89-9052-20d2a247c180 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:47.341062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c669b-8da1-4f89-9052-20d2a247c180 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:47.341565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c669b-8da1-4f89-9052-20d2a247c180 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:47.358855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c669b-8da1-4f89-9052-20d2a247c180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:47.360479 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c669b-8da1-4f89-9052-20d2a247c180 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:47.381266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-2e8c669b-8da1-4f89-9052-20d2a247c180 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:47.381266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:47.381266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:47.381266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:47.381266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:47.381266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:47.381266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:47.381266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:47.381266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:47.381266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:47.381266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:47.381266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:47.381266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:47.381266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:47.381266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:47.382361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:47.382361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:47.382361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:47.382361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:47.382361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:47.382361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:47.382361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:47.382361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:47.382361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:47.382361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:47.382361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:47.382361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:47.382361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:47.382361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:47.382361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:47.382361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:47.382361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:47.383340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:47.383340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:47.383340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:47.383340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:47.383340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 246/515] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:47.383728 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c669b-8da1-4f89-9052-20d2a247c180 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:47.391712 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb210de-4e95-470e-a158-c9e122b74004 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:47.421520 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:47.427202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfe1fc-a043-4fce-9d92-6e397b48a596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:47.428010 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfe1fc-a043-4fce-9d92-6e397b48a596 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:47.428282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfe1fc-a043-4fce-9d92-6e397b48a596 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:47.428552 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfe1fc-a043-4fce-9d92-6e397b48a596 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:47.437466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bfe1fc-a043-4fce-9d92-6e397b48a596 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:47.442851 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bfe1fc-a043-4fce-9d92-6e397b48a596 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:47.454372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-54bfe1fc-a043-4fce-9d92-6e397b48a596 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 16 14:49:47.454372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:47.454372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:47.454372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:47.454372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:47.454372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:47.454372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:47.454372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:47.454372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:47.454372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:47.454372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:47.454372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:47.454372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:49:47.454372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:49:47.454372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:49:47.454372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:49:47.455401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:47.455401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:47.455401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:49:47.455401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:49:47.455401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:49:47.455401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:49:47.455401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:47.455401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:47.455401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:47.455401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:47.455401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:47.455401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:47.455401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:47.455401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:47.455401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:47.455401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:47.455401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:47.455401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:47.456295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:47.456295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:47.456295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:49:47.456295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:49:47.456295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:49:47.456295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:49:47.456295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:49:47.456295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:49:47.456295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 16 14:49:47.456295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:47.456295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 270/516] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:47.456883 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfe1fc-a043-4fce-9d92-6e397b48a596 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:47.504711 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c669b-8da1-4f89-9052-20d2a247c180 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:47.557327 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b4534-fb5d-4ec5-a968-dd6ea7615ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:47.558321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b4534-fb5d-4ec5-a968-dd6ea7615ca8 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:47.558691 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b4534-fb5d-4ec5-a968-dd6ea7615ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:47.559187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b4534-fb5d-4ec5-a968-dd6ea7615ca8 None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:47.560052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b4534-fb5d-4ec5-a968-dd6ea7615ca8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:47.561802 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b4534-fb5d-4ec5-a968-dd6ea7615ca8 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:47.596310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfe1fc-a043-4fce-9d92-6e397b48a596 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:47.638593 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee375828-5158-40bc-94d4-b3948b812ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:47.639281 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee375828-5158-40bc-94d4-b3948b812ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:47.639560 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee375828-5158-40bc-94d4-b3948b812ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:47.639836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee375828-5158-40bc-94d4-b3948b812ef5 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:47.640571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee375828-5158-40bc-94d4-b3948b812ef5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:47.640866 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee375828-5158-40bc-94d4-b3948b812ef5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:47.642252 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee375828-5158-40bc-94d4-b3948b812ef5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:47.654964 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 271/517] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:47.667164 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 247/518] 10.222.0.27 () {60 vars in 1132 bytes} [Sat Jul 16 14:49:47 2022] POST /identity/v3/users => generated 247 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:49:47.667402 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee375828-5158-40bc-94d4-b3948b812ef5 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:47.667500 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b4534-fb5d-4ec5-a968-dd6ea7615ca8 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:47.706576 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:47.747144 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:47.828390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee375828-5158-40bc-94d4-b3948b812ef5 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:47.844785 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b4534-fb5d-4ec5-a968-dd6ea7615ca8 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:47.893442 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7aef3-d3da-4515-b16e-9eb437102341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:47.894495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7aef3-d3da-4515-b16e-9eb437102341 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:47.894790 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7aef3-d3da-4515-b16e-9eb437102341 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:47.895253 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7aef3-d3da-4515-b16e-9eb437102341 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:47.895973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7aef3-d3da-4515-b16e-9eb437102341 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:47.896362 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7aef3-d3da-4515-b16e-9eb437102341 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:47.897860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7aef3-d3da-4515-b16e-9eb437102341 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:47.899024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2c5cf-138a-47cc-8c78-bb083ecc0da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:47.900599 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2c5cf-138a-47cc-8c78-bb083ecc0da0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:47.901843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2c5cf-138a-47cc-8c78-bb083ecc0da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:47.902337 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2c5cf-138a-47cc-8c78-bb083ecc0da0 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:47.903413 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf2c5cf-138a-47cc-8c78-bb083ecc0da0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:47.903726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf2c5cf-138a-47cc-8c78-bb083ecc0da0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:47.907010 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf2c5cf-138a-47cc-8c78-bb083ecc0da0 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:47.911975 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 248/519] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:47.913099 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7aef3-d3da-4515-b16e-9eb437102341 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:47.933961 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 272/520] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:47.935663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2c5cf-138a-47cc-8c78-bb083ecc0da0 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:48.020928 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:48.042253 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7aef3-d3da-4515-b16e-9eb437102341 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:48.096660 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bce910-6207-481e-9e6e-b991bf633def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:48.097279 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bce910-6207-481e-9e6e-b991bf633def None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:48.097563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bce910-6207-481e-9e6e-b991bf633def None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:48.097837 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bce910-6207-481e-9e6e-b991bf633def None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:48.097931 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2c5cf-138a-47cc-8c78-bb083ecc0da0 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:48.099119 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bce910-6207-481e-9e6e-b991bf633def None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:48.099433 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bce910-6207-481e-9e6e-b991bf633def None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:48.103460 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bce910-6207-481e-9e6e-b991bf633def None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:48.111316 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:48.128732 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 249/521] 10.222.0.27 () {56 vars in 1137 bytes} [Sat Jul 16 14:49:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:48.136765 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bce910-6207-481e-9e6e-b991bf633def None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:48.156951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f05e1-5ad0-4c3b-b434-408438b5f8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:48.158392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f05e1-5ad0-4c3b-b434-408438b5f8fc None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:48.158968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f05e1-5ad0-4c3b-b434-408438b5f8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:48.159411 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f05e1-5ad0-4c3b-b434-408438b5f8fc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:48.172951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f05e1-5ad0-4c3b-b434-408438b5f8fc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:48.175586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f05e1-5ad0-4c3b-b434-408438b5f8fc None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:48.203618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-622f05e1-5ad0-4c3b-b434-408438b5f8fc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 16 14:49:48.203618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:48.203618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:48.203618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:48.203618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:48.203618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:48.203618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:48.203618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:48.203618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:48.203618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:48.203618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:48.203618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:48.203618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:49:48.203618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:49:48.203618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:49:48.203618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:49:48.204571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:48.204571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:48.204571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:49:48.204571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:49:48.204571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:49:48.204571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:49:48.204571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:48.204571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:48.204571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:48.204571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:48.204571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:48.204571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:48.204571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:48.204571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:48.204571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:48.204571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:48.204571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:48.204571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:48.205451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:48.205451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:48.205451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:49:48.205451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:49:48.205451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:49:48.205451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:49:48.205451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:49:48.205451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:49:48.205451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 16 14:49:48.205451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:48.205966 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 273/522] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:49:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:48.208586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f05e1-5ad0-4c3b-b434-408438b5f8fc None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:48.277215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bce910-6207-481e-9e6e-b991bf633def None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:48.349569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13416aca-d4b4-4879-b7e0-02f4221dec80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:48.350947 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13416aca-d4b4-4879-b7e0-02f4221dec80 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:48.351512 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13416aca-d4b4-4879-b7e0-02f4221dec80 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:48.352481 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13416aca-d4b4-4879-b7e0-02f4221dec80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:48.365035 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13416aca-d4b4-4879-b7e0-02f4221dec80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:48.368083 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13416aca-d4b4-4879-b7e0-02f4221dec80 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:48.372526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:48.373328 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f05e1-5ad0-4c3b-b434-408438b5f8fc None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:48.396260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-13416aca-d4b4-4879-b7e0-02f4221dec80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:49:48.396260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:48.396260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:48.396260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:48.396260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:48.396260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:48.396260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:48.396260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:48.396260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:48.396260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:48.396260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:48.396260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:48.396260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:49:48.396260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:49:48.396260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:49:48.398166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:49:48.398166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:48.398166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:48.398166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:48.398166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:48.398166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:48.398166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:48.398166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:48.398166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:48.398166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:48.398166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:48.398166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:48.398166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:48.398166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:48.398166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:48.398166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:48.398166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:48.400654 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:49:48.400654 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:49:48.400654 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:49:48.400654 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:49:48.400654 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:49:48.400654 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:49:48.400654 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:49:48.400654 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:48.400654 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 250/523] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:49:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:48.401605 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13416aca-d4b4-4879-b7e0-02f4221dec80 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:48.431880 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b3616-1e3c-4643-a1d0-dffd3cc5fd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:48.433107 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b3616-1e3c-4643-a1d0-dffd3cc5fd15 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:48.433425 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b3616-1e3c-4643-a1d0-dffd3cc5fd15 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:48.433695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b3616-1e3c-4643-a1d0-dffd3cc5fd15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:48.443163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b3616-1e3c-4643-a1d0-dffd3cc5fd15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:48.444627 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245b3616-1e3c-4643-a1d0-dffd3cc5fd15 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:48.474261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-245b3616-1e3c-4643-a1d0-dffd3cc5fd15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:49:48.474261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:48.474261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:48.474261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:48.474261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:48.474261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:48.474261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:48.474261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:48.474261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:48.474261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:48.474261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:48.474261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:48.474261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:49:48.474261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:49:48.474261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:49:48.475695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:49:48.475695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:48.475695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:48.475695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:48.475695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:48.475695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:48.475695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:48.475695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:48.475695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:48.475695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:48.475695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:48.475695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:48.475695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:48.475695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:48.475695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:48.475695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:48.475695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:48.477217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:49:48.477217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:49:48.477217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:49:48.477217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:49:48.477217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:49:48.477217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:49:48.477217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:49:48.477217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:48.477918 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 274/524] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:49:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:48.479340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b3616-1e3c-4643-a1d0-dffd3cc5fd15 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:48.533686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:48.594876 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13416aca-d4b4-4879-b7e0-02f4221dec80 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:48.656728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b3616-1e3c-4643-a1d0-dffd3cc5fd15 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:48.659044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfddb5a-7dae-4620-aa5c-8f31333b50a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:48.660110 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfddb5a-7dae-4620-aa5c-8f31333b50a6 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:48.660593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfddb5a-7dae-4620-aa5c-8f31333b50a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:48.661055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfddb5a-7dae-4620-aa5c-8f31333b50a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:48.700557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529a807-f945-49d4-a149-94ebc8222ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:48.701247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529a807-f945-49d4-a149-94ebc8222ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:48.701520 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529a807-f945-49d4-a149-94ebc8222ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:48.701794 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529a807-f945-49d4-a149-94ebc8222ee6 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:48.702663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529a807-f945-49d4-a149-94ebc8222ee6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:48.703000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529a807-f945-49d4-a149-94ebc8222ee6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:48.704481 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529a807-f945-49d4-a149-94ebc8222ee6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:48.734043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 275/525] 10.222.0.27 () {56 vars in 1139 bytes} [Sat Jul 16 14:49:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:48.734043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529a807-f945-49d4-a149-94ebc8222ee6 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:48.799241 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:48.805235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfddb5a-7dae-4620-aa5c-8f31333b50a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:48.809637 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfddb5a-7dae-4620-aa5c-8f31333b50a6 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:48.858766 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:48.886877 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529a807-f945-49d4-a149-94ebc8222ee6 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:48.954760 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcff68-8ffa-4d69-904f-f52f340e19e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:48.955749 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcff68-8ffa-4d69-904f-f52f340e19e2 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:48.956201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcff68-8ffa-4d69-904f-f52f340e19e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:48.956669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcff68-8ffa-4d69-904f-f52f340e19e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:48.958131 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcff68-8ffa-4d69-904f-f52f340e19e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:48.958675 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcff68-8ffa-4d69-904f-f52f340e19e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:48.962176 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcff68-8ffa-4d69-904f-f52f340e19e2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:48.972728 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 251/526] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:49:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:49:48.972728 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfddb5a-7dae-4620-aa5c-8f31333b50a6 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:49.000528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 276/527] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:49:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:49.000528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcff68-8ffa-4d69-904f-f52f340e19e2 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:49.135053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcff68-8ffa-4d69-904f-f52f340e19e2 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:49.162600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:49.167949 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfddb5a-7dae-4620-aa5c-8f31333b50a6 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:49.180179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d989f2-22b0-4b7a-b31e-cbb90d41b90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:49.181141 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d989f2-22b0-4b7a-b31e-cbb90d41b90f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:49.181550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d989f2-22b0-4b7a-b31e-cbb90d41b90f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:49.181953 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d989f2-22b0-4b7a-b31e-cbb90d41b90f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:49.206362 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:49.235659 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50305bc0-8d12-458d-b17f-9289beaa6d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:49.236712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50305bc0-8d12-458d-b17f-9289beaa6d6f None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:49.237166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50305bc0-8d12-458d-b17f-9289beaa6d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:49.237650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50305bc0-8d12-458d-b17f-9289beaa6d6f None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:49.238845 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50305bc0-8d12-458d-b17f-9289beaa6d6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:49.239295 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50305bc0-8d12-458d-b17f-9289beaa6d6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:49.240898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50305bc0-8d12-458d-b17f-9289beaa6d6f None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:49.263629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 252/528] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:49:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:49.263629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50305bc0-8d12-458d-b17f-9289beaa6d6f None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:49.312237 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d989f2-22b0-4b7a-b31e-cbb90d41b90f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:49.314542 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d989f2-22b0-4b7a-b31e-cbb90d41b90f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:49.421212 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50305bc0-8d12-458d-b17f-9289beaa6d6f None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:49.471521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587284a-97da-4bf7-b36a-292a9b7b2dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:49.472254 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587284a-97da-4bf7-b36a-292a9b7b2dfa None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:49.472536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587284a-97da-4bf7-b36a-292a9b7b2dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:49.472817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587284a-97da-4bf7-b36a-292a9b7b2dfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:49.487760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587284a-97da-4bf7-b36a-292a9b7b2dfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:49.489260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587284a-97da-4bf7-b36a-292a9b7b2dfa None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:49.511770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-3587284a-97da-4bf7-b36a-292a9b7b2dfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:49.511770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:49.511770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:49.511770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:49.511770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:49.511770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:49.511770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:49.511770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:49.511770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:49.511770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:49.511770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:49.511770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:49.511770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:49.511770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:49.511770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:49.513527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:49.513527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:49.513527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:49.513527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:49.513527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:49.513527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:49.513527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:49.513527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:49.513527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:49.513527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:49.513527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:49.513527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:49.513527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:49.513527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:49.513527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:49.513527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:49.513527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:49.515287 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:49.515287 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:49.515287 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:49.515287 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:49.515287 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 253/529] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:49.515287 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587284a-97da-4bf7-b36a-292a9b7b2dfa None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:49.535484 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:49.538306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 277/530] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:49:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:49:49.540109 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d989f2-22b0-4b7a-b31e-cbb90d41b90f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:49.604523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:49.693279 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d989f2-22b0-4b7a-b31e-cbb90d41b90f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:49.709962 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587284a-97da-4bf7-b36a-292a9b7b2dfa None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:49.752502 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874739dc-4e1f-49e5-af5f-fbf4fb3bb9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:49.753814 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874739dc-4e1f-49e5-af5f-fbf4fb3bb9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:49.754495 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874739dc-4e1f-49e5-af5f-fbf4fb3bb9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:49.755001 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874739dc-4e1f-49e5-af5f-fbf4fb3bb9b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:49.756589 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874739dc-4e1f-49e5-af5f-fbf4fb3bb9b3 None admin] RBAC: Adding query filter params (role.id=21119ebd056740048b9ec639ebb20446, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=3f2e241494244edf8d283e8f44192c46) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:49.757147 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874739dc-4e1f-49e5-af5f-fbf4fb3bb9b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:49.761780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874739dc-4e1f-49e5-af5f-fbf4fb3bb9b3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:49.774421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90ff17-55b1-4bbb-b0bf-018447d97136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:49.775221 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 278/531] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:49:49 2022] GET /identity/v3/role_assignments?user.id=3f2e241494244edf8d283e8f44192c46&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=21119ebd056740048b9ec639ebb20446 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:49.775579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90ff17-55b1-4bbb-b0bf-018447d97136 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:49.776092 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90ff17-55b1-4bbb-b0bf-018447d97136 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:49.776597 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90ff17-55b1-4bbb-b0bf-018447d97136 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:49.777003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874739dc-4e1f-49e5-af5f-fbf4fb3bb9b3 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:49.778265 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90ff17-55b1-4bbb-b0bf-018447d97136 None admin] RBAC: Adding query filter params (role.id=21119ebd056740048b9ec639ebb20446, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=7d689162291f4f4b97e886ea20d3a8b2) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:49.778829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90ff17-55b1-4bbb-b0bf-018447d97136 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:49.781753 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90ff17-55b1-4bbb-b0bf-018447d97136 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:49.794442 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 254/532] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:49:49 2022] GET /identity/v3/role_assignments?user.id=7d689162291f4f4b97e886ea20d3a8b2&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=21119ebd056740048b9ec639ebb20446 => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:49.795971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90ff17-55b1-4bbb-b0bf-018447d97136 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:49.918058 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874739dc-4e1f-49e5-af5f-fbf4fb3bb9b3 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:49.946355 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:49.971201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d008e46-e288-4f3a-bd9a-cdbe10b6f92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:49.972178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:49.972416 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d008e46-e288-4f3a-bd9a-cdbe10b6f92b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:49.973031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d008e46-e288-4f3a-bd9a-cdbe10b6f92b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:49.973287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d008e46-e288-4f3a-bd9a-cdbe10b6f92b None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:49.975973 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d008e46-e288-4f3a-bd9a-cdbe10b6f92b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:49.975973 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d008e46-e288-4f3a-bd9a-cdbe10b6f92b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:49.977605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d008e46-e288-4f3a-bd9a-cdbe10b6f92b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:50.000584 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 279/533] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:50.010376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3dcf9-8ef0-4163-ab11-134b0428c149 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:50.010376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3dcf9-8ef0-4163-ab11-134b0428c149 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:50.010376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3dcf9-8ef0-4163-ab11-134b0428c149 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:50.012538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90ff17-55b1-4bbb-b0bf-018447d97136 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:50.014084 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 280/534] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:50 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:50.015656 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3dcf9-8ef0-4163-ab11-134b0428c149 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:50.066300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d6d46-94b0-4259-98d4-0f5c10b29f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:50.067918 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d6d46-94b0-4259-98d4-0f5c10b29f24 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:50.068551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d6d46-94b0-4259-98d4-0f5c10b29f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:50.069570 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d6d46-94b0-4259-98d4-0f5c10b29f24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:50.087855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d6d46-94b0-4259-98d4-0f5c10b29f24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:50.089881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d6d46-94b0-4259-98d4-0f5c10b29f24 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:50.113625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-af0d6d46-94b0-4259-98d4-0f5c10b29f24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:50.113625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:50.113625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:50.113625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:50.113625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:50.113625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:50.113625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:50.113625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:50.113625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:50.113625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:50.113625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:50.113625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:50.113625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:50.113625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:50.113625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:50.114565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:50.114565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:50.114565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:50.114565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:50.114565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:50.114565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:50.114565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:50.114565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:50.114565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:50.114565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:50.114565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:50.114565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:50.114565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:50.114565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:50.114565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:50.114565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:50.114565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:50.115460 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:50.115460 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:50.115460 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:50.115460 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:50.120248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 255/535] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:50.123134 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420d57d-d83d-4b0d-945c-67d00dc1a94b None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:50.123134 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420d57d-d83d-4b0d-945c-67d00dc1a94b None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:50.123134 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420d57d-d83d-4b0d-945c-67d00dc1a94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:50.141185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-a420d57d-d83d-4b0d-945c-67d00dc1a94b None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:50.142455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a420d57d-d83d-4b0d-945c-67d00dc1a94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:50.189426 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3dcf9-8ef0-4163-ab11-134b0428c149 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:50.246796 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce959116-3ef7-4cac-a900-5a84cc15fc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:50.248877 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce959116-3ef7-4cac-a900-5a84cc15fc6d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:50.250304 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce959116-3ef7-4cac-a900-5a84cc15fc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:50.251435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce959116-3ef7-4cac-a900-5a84cc15fc6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:50.267125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce959116-3ef7-4cac-a900-5a84cc15fc6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:50.270297 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce959116-3ef7-4cac-a900-5a84cc15fc6d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:50.279246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 256/536] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:50.280283 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420d57d-d83d-4b0d-945c-67d00dc1a94b None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:50.300904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-ce959116-3ef7-4cac-a900-5a84cc15fc6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:49:50.300904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:50.300904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:50.300904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:50.300904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:50.300904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:50.300904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:50.300904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:50.300904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:50.300904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:50.300904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:50.300904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:50.300904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:49:50.300904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:49:50.300904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:49:50.302560 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:49:50.302560 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:50.302560 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:50.302560 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:50.302560 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:50.302560 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:50.302560 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:50.302560 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:50.302560 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:50.302560 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:50.302560 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:50.302560 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:50.302560 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:50.302560 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:50.302560 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:50.302560 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:50.302560 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:50.304246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:49:50.304246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:49:50.304246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:49:50.304246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:49:50.304246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:49:50.304246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:49:50.304246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:49:50.304246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:50.315703 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 281/537] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:49:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:50.322624 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce959116-3ef7-4cac-a900-5a84cc15fc6d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:50.377590 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:50.391717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:50.439378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420d57d-d83d-4b0d-945c-67d00dc1a94b None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:50.480092 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985bd9da-293d-4bac-bfc6-a315e456db6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:50.481955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985bd9da-293d-4bac-bfc6-a315e456db6e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:50.482262 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985bd9da-293d-4bac-bfc6-a315e456db6e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:50.482503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985bd9da-293d-4bac-bfc6-a315e456db6e None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:50.483272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985bd9da-293d-4bac-bfc6-a315e456db6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:50.483575 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985bd9da-293d-4bac-bfc6-a315e456db6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:50.485382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985bd9da-293d-4bac-bfc6-a315e456db6e None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:50.500069 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 257/538] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:50.501079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985bd9da-293d-4bac-bfc6-a315e456db6e None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:50.522080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce959116-3ef7-4cac-a900-5a84cc15fc6d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:50.587722 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624393e-6570-461b-81ef-b26341b59b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:50.589320 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624393e-6570-461b-81ef-b26341b59b92 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:50.590107 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624393e-6570-461b-81ef-b26341b59b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:50.590595 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624393e-6570-461b-81ef-b26341b59b92 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:50.592251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624393e-6570-461b-81ef-b26341b59b92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:50.592794 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624393e-6570-461b-81ef-b26341b59b92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:50.595629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624393e-6570-461b-81ef-b26341b59b92 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:50.627115 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 282/539] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:49:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:50.628813 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624393e-6570-461b-81ef-b26341b59b92 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:50.631391 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985bd9da-293d-4bac-bfc6-a315e456db6e None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:50.660530 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:50.682506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307612f9-5d37-4f12-b195-98b602ba90f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:50.683695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307612f9-5d37-4f12-b195-98b602ba90f3 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:50.684056 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307612f9-5d37-4f12-b195-98b602ba90f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:50.684403 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307612f9-5d37-4f12-b195-98b602ba90f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:50.699760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307612f9-5d37-4f12-b195-98b602ba90f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:50.709926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-307612f9-5d37-4f12-b195-98b602ba90f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:49:50.709926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:50.709926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:50.709926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:50.709926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:50.709926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:50.709926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:50.709926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:50.709926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:50.709926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:50.709926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:50.709926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:50.709926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:50.709926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:50.709926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:50.709926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:50.711613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:50.711613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:50.711613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:50.711613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:50.711613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:50.711613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:50.711613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:50.711613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:50.711613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:50.711613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:50.711613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:50.711613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:50.711613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:50.711613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:50.711613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:50.711613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:50.713414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:50.713414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:50.713414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:50.713414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:50.713414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:50.713414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:50.713414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:50.713414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:50.713414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:50.713414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:50.713414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:49:50.713414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:50.713414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 258/540] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:49:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:50.714915 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307612f9-5d37-4f12-b195-98b602ba90f3 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:50.805719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:50.815400 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624393e-6570-461b-81ef-b26341b59b92 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:50.857144 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10599b-5fca-4941-b972-23427a23fc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:50.858995 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10599b-5fca-4941-b972-23427a23fc32 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:50.858995 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10599b-5fca-4941-b972-23427a23fc32 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:50.858995 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10599b-5fca-4941-b972-23427a23fc32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:50.866813 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b10599b-5fca-4941-b972-23427a23fc32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:50.868980 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b10599b-5fca-4941-b972-23427a23fc32 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:50.884336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-2b10599b-5fca-4941-b972-23427a23fc32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:49:50.884336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:50.884336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:50.884336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:50.884336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:50.884336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:50.884336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:50.884336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:50.884336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:50.884336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:50.884336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:50.884336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:50.884336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:49:50.884336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:49:50.884336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:49:50.886309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:49:50.886309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:50.886309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:50.886309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:50.886309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:50.886309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:50.886309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:50.886309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:50.886309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:50.886309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:50.886309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:50.886309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:50.886309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:50.886309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:50.886309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:50.886309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:50.886309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:50.888437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:49:50.888437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:49:50.888437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:49:50.888437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:49:50.888437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:49:50.888437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:49:50.888437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:49:50.888437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:50.888437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 283/541] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:49:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:50.888437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10599b-5fca-4941-b972-23427a23fc32 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:50.951594 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307612f9-5d37-4f12-b195-98b602ba90f3 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:51.010559 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b10599b-5fca-4941-b972-23427a23fc32 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:51.026574 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf80ca-ef6b-4797-8af9-ac3d5a65d266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:51.028868 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf80ca-ef6b-4797-8af9-ac3d5a65d266 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:51.029622 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf80ca-ef6b-4797-8af9-ac3d5a65d266 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:51.030268 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf80ca-ef6b-4797-8af9-ac3d5a65d266 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:51.032439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf80ca-ef6b-4797-8af9-ac3d5a65d266 None admin] RBAC: Adding query filter params (role.id=21119ebd056740048b9ec639ebb20446, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=8c21d12c977e4c36b4f0ed442b11ef49) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:51.033199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf80ca-ef6b-4797-8af9-ac3d5a65d266 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:51.036213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf80ca-ef6b-4797-8af9-ac3d5a65d266 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:51.051396 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 259/542] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:49:50 2022] GET /identity/v3/role_assignments?user.id=8c21d12c977e4c36b4f0ed442b11ef49&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=21119ebd056740048b9ec639ebb20446 => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:51.051396 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf80ca-ef6b-4797-8af9-ac3d5a65d266 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:51.074245 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f0477-637d-4fd3-a653-58463587ee55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:51.074835 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f0477-637d-4fd3-a653-58463587ee55 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:51.075153 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f0477-637d-4fd3-a653-58463587ee55 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:51.075432 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f0477-637d-4fd3-a653-58463587ee55 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:51.076212 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f0477-637d-4fd3-a653-58463587ee55 None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:51.076526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f0477-637d-4fd3-a653-58463587ee55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:51.077978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f0477-637d-4fd3-a653-58463587ee55 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:51.093762 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 284/543] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:49:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:51.097851 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f0477-637d-4fd3-a653-58463587ee55 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:51.131449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:51.146678 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:51.273864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf80ca-ef6b-4797-8af9-ac3d5a65d266 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:51.308016 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f0477-637d-4fd3-a653-58463587ee55 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:51.355896 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10af7f8-998d-43b7-817d-bff072eea03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:51.356929 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10af7f8-998d-43b7-817d-bff072eea03e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:51.357585 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10af7f8-998d-43b7-817d-bff072eea03e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:51.358482 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10af7f8-998d-43b7-817d-bff072eea03e None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:51.359946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10af7f8-998d-43b7-817d-bff072eea03e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:51.360739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10af7f8-998d-43b7-817d-bff072eea03e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:51.363225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10af7f8-998d-43b7-817d-bff072eea03e None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:51.366240 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e84406-3282-4d8d-b5dc-4352e082a309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:51.367688 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e84406-3282-4d8d-b5dc-4352e082a309 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:51.368129 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e84406-3282-4d8d-b5dc-4352e082a309 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:51.368418 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e84406-3282-4d8d-b5dc-4352e082a309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:51.387817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 260/544] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:49:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:51.391750 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e84406-3282-4d8d-b5dc-4352e082a309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:51.393995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10af7f8-998d-43b7-817d-bff072eea03e None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:51.394207 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e84406-3282-4d8d-b5dc-4352e082a309 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:51.417982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-c9e84406-3282-4d8d-b5dc-4352e082a309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:51.417982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:51.417982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:51.417982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:51.417982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:51.417982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:51.417982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:51.417982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:51.417982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:51.417982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:51.417982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:51.417982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:51.417982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:51.417982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:51.417982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:51.418981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:51.418981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:51.418981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:51.418981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:51.418981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:51.418981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:51.418981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:51.418981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:51.418981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:51.418981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:51.418981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:51.418981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:51.418981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:51.418981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:51.418981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:51.418981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:51.418981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:51.419893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:51.419893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:51.419893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:51.419893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:51.423011 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 285/545] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:51.432299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e84406-3282-4d8d-b5dc-4352e082a309 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:51.537329 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:51.615604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e84406-3282-4d8d-b5dc-4352e082a309 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:51.635968 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10af7f8-998d-43b7-817d-bff072eea03e None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:51.637689 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:51.698538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084b6e1-855b-492b-97da-a1a06b4f205e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:51.699356 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084b6e1-855b-492b-97da-a1a06b4f205e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:51.699642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084b6e1-855b-492b-97da-a1a06b4f205e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:51.699919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084b6e1-855b-492b-97da-a1a06b4f205e None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:51.700669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8084b6e1-855b-492b-97da-a1a06b4f205e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:51.700978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8084b6e1-855b-492b-97da-a1a06b4f205e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:51.702403 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8084b6e1-855b-492b-97da-a1a06b4f205e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:51.705622 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911f64f-7f22-4c48-8b98-61485d0234cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:51.706310 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911f64f-7f22-4c48-8b98-61485d0234cd None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:51.706591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911f64f-7f22-4c48-8b98-61485d0234cd None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:51.706868 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911f64f-7f22-4c48-8b98-61485d0234cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:51.707750 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911f64f-7f22-4c48-8b98-61485d0234cd None admin] RBAC: Adding query filter params (role.id=21119ebd056740048b9ec639ebb20446, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=6e372cc62af24664b01a967435eb8902) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:51.708065 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911f64f-7f22-4c48-8b98-61485d0234cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:51.709632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911f64f-7f22-4c48-8b98-61485d0234cd None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:51.723114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 261/546] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:49:51 2022] GET /identity/v3/role_assignments?user.id=6e372cc62af24664b01a967435eb8902&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=21119ebd056740048b9ec639ebb20446 => generated 259 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:49:51.730508 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 286/547] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:51.736085 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911f64f-7f22-4c48-8b98-61485d0234cd None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:51.921315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911f64f-7f22-4c48-8b98-61485d0234cd None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:51.970728 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2666f8a-0717-464a-8449-21ba69241a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:51.971608 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2666f8a-0717-464a-8449-21ba69241a43 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:51.971879 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2666f8a-0717-464a-8449-21ba69241a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:51.972153 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2666f8a-0717-464a-8449-21ba69241a43 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:51.981677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2666f8a-0717-464a-8449-21ba69241a43 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:51.983171 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2666f8a-0717-464a-8449-21ba69241a43 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:52.001187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-f2666f8a-0717-464a-8449-21ba69241a43 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 16 14:49:52.001187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:52.001187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:52.001187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:52.001187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:52.001187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:52.001187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:52.001187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:52.001187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:52.001187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:52.001187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:52.001187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:52.001187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:49:52.001187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:49:52.001187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:49:52.001187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:49:52.003503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:52.003503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:52.003503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:49:52.003503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:49:52.003503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:49:52.003503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:49:52.003503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:52.003503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:52.003503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:52.003503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:52.003503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:52.003503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:52.003503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:52.003503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:52.003503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:52.003503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:52.003503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:52.003503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:52.005569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:52.005569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:52.005569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:49:52.005569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:49:52.005569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:49:52.005569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:49:52.005569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:49:52.005569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:49:52.005569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 16 14:49:52.005569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:52.005569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 262/548] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:49:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:52.009988 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084b6e1-855b-492b-97da-a1a06b4f205e None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:52.239215 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084b6e1-855b-492b-97da-a1a06b4f205e None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:52.297412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:52.320236 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb05e02-a565-4442-ae0a-3f9d01eb5dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:52.321395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb05e02-a565-4442-ae0a-3f9d01eb5dca None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:52.321891 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb05e02-a565-4442-ae0a-3f9d01eb5dca None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:52.322381 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb05e02-a565-4442-ae0a-3f9d01eb5dca None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:52.324009 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb05e02-a565-4442-ae0a-3f9d01eb5dca None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:52.324559 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb05e02-a565-4442-ae0a-3f9d01eb5dca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:52.327365 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb05e02-a565-4442-ae0a-3f9d01eb5dca None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:52.377014 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 287/549] 10.222.0.27 () {56 vars in 1135 bytes} [Sat Jul 16 14:49:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:49:52.383298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2666f8a-0717-464a-8449-21ba69241a43 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:52.441053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:52.510523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a02f6-072e-4cc6-967f-968291f9f996 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:52.510982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a02f6-072e-4cc6-967f-968291f9f996 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:52.511478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a02f6-072e-4cc6-967f-968291f9f996 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:52.514360 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 288/550] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:52.526732 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8d629-e327-40d2-a67f-ef4d64daa9fe None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:52.527256 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8d629-e327-40d2-a67f-ef4d64daa9fe None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:52.527749 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8d629-e327-40d2-a67f-ef4d64daa9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:52.545717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2666f8a-0717-464a-8449-21ba69241a43 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:52.569513 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-61a8d629-e327-40d2-a67f-ef4d64daa9fe None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:52.574690 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-61a8d629-e327-40d2-a67f-ef4d64daa9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:52.588299 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a7a94-6488-499e-9359-33e95188e601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:52.588979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a7a94-6488-499e-9359-33e95188e601 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:52.589267 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a7a94-6488-499e-9359-33e95188e601 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:52.589549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a7a94-6488-499e-9359-33e95188e601 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:52.671945 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a7a94-6488-499e-9359-33e95188e601 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:52.673460 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a7a94-6488-499e-9359-33e95188e601 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:52.758649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 289/551] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:52.788288 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:52.794187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8d629-e327-40d2-a67f-ef4d64daa9fe None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:52.974792 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 263/552] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:49:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:49:52.977224 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370264c-466d-45e1-97a6-0842c1bf8578 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:52.977533 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370264c-466d-45e1-97a6-0842c1bf8578 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:52.977847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b370264c-466d-45e1-97a6-0842c1bf8578 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:52.983617 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 264/553] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:52.984566 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370264c-466d-45e1-97a6-0842c1bf8578 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:53.021661 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8d629-e327-40d2-a67f-ef4d64daa9fe None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:53.043808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:53.068169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5fdb49-e4bb-4a57-9d8f-01b3f8e63e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:53.069062 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fdb49-e4bb-4a57-9d8f-01b3f8e63e2e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:53.069341 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fdb49-e4bb-4a57-9d8f-01b3f8e63e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:53.069615 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5fdb49-e4bb-4a57-9d8f-01b3f8e63e2e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:53.082330 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5fdb49-e4bb-4a57-9d8f-01b3f8e63e2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:53.092291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-ac5fdb49-e4bb-4a57-9d8f-01b3f8e63e2e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:49:53.092291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:53.092291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:53.092291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:53.092291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:53.092291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:53.092291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:53.092291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:53.092291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:53.092291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:53.092291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:53.092291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:53.092291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:53.092291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:53.092291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:53.092291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:53.093265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:53.093265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:53.093265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:53.093265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:53.093265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:53.093265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:53.093265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:53.093265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:53.093265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:53.093265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:53.093265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:53.093265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:53.093265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:53.093265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:53.093265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:53.093265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:53.094068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:53.094068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:53.094068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:53.094068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:53.094068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:53.094068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:53.094068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:53.094068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:53.094068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:53.094068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:53.094068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:49:53.094068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:53.094068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 290/554] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:49:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:53.095268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944883d-a7e1-4fe0-8bbe-8a144194c5f6 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:53.095528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944883d-a7e1-4fe0-8bbe-8a144194c5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:53.095814 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944883d-a7e1-4fe0-8bbe-8a144194c5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:53.114255 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-6944883d-a7e1-4fe0-8bbe-8a144194c5f6 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:53.115515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6944883d-a7e1-4fe0-8bbe-8a144194c5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:53.175284 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b370264c-466d-45e1-97a6-0842c1bf8578 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:53.194447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 291/555] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:53.195639 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944883d-a7e1-4fe0-8bbe-8a144194c5f6 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:53.248744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:53.260101 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2052442c-e520-42f3-91fd-eb0dfd45c524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:53.261850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052442c-e520-42f3-91fd-eb0dfd45c524 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:53.261850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052442c-e520-42f3-91fd-eb0dfd45c524 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:53.262137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052442c-e520-42f3-91fd-eb0dfd45c524 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:53.276514 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2052442c-e520-42f3-91fd-eb0dfd45c524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:53.278001 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2052442c-e520-42f3-91fd-eb0dfd45c524 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:53.303006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-2052442c-e520-42f3-91fd-eb0dfd45c524 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:53.303006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:53.303006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:53.303006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:53.303006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:53.303006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:53.303006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:53.303006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:53.303006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:53.303006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:53.303006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:53.303006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:53.303006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:53.303006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:53.303006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:53.305272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:53.305272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:53.305272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:53.305272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:53.305272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:53.305272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:53.305272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:53.305272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:53.305272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:53.305272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:53.305272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:53.305272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:53.305272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:53.305272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:53.305272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:53.305272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:53.305272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:53.307562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:53.307562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:53.307562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:53.307562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:53.307562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 265/556] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:53.307562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2052442c-e520-42f3-91fd-eb0dfd45c524 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:53.374664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944883d-a7e1-4fe0-8bbe-8a144194c5f6 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:53.379442 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:53.423120 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509a83c-18ae-446f-8ca7-d15f16ad38ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:53.424222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509a83c-18ae-446f-8ca7-d15f16ad38ad None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:53.424711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509a83c-18ae-446f-8ca7-d15f16ad38ad None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:53.425221 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509a83c-18ae-446f-8ca7-d15f16ad38ad None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:53.426207 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509a83c-18ae-446f-8ca7-d15f16ad38ad None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:53.426728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509a83c-18ae-446f-8ca7-d15f16ad38ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:53.428394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509a83c-18ae-446f-8ca7-d15f16ad38ad None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:53.439262 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 292/557] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:49:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:53.440569 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509a83c-18ae-446f-8ca7-d15f16ad38ad None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:53.452272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2052442c-e520-42f3-91fd-eb0dfd45c524 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:53.507508 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16447cfd-10dd-4de5-997a-da8194126075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:53.509171 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16447cfd-10dd-4de5-997a-da8194126075 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:53.509921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16447cfd-10dd-4de5-997a-da8194126075 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:53.510531 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16447cfd-10dd-4de5-997a-da8194126075 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:53.527607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16447cfd-10dd-4de5-997a-da8194126075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:53.536590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-16447cfd-10dd-4de5-997a-da8194126075 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:49:53.536590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:53.536590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:53.536590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:53.536590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:53.536590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:53.536590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:53.536590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:53.536590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:53.536590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:53.536590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:53.536590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:53.536590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:53.536590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:53.536590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:53.536590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:53.538044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:53.538044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:53.538044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:53.538044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:53.538044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:53.538044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:53.538044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:53.538044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:53.538044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:53.538044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:53.538044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:53.538044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:53.538044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:53.538044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:53.538044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:53.538044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:53.539513 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:53.539513 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:53.539513 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:53.539513 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:53.539513 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:53.539513 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:53.539513 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:53.539513 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:53.539513 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:53.539513 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:53.539513 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:49:53.539513 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:53.548030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 266/558] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:49:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:53.549534 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16447cfd-10dd-4de5-997a-da8194126075 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:53.620992 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509a83c-18ae-446f-8ca7-d15f16ad38ad None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:53.651512 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:53.667630 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67eedf0-af7b-4a07-8f9d-8a45fcd32fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:53.668861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67eedf0-af7b-4a07-8f9d-8a45fcd32fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:53.669361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67eedf0-af7b-4a07-8f9d-8a45fcd32fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:53.669806 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67eedf0-af7b-4a07-8f9d-8a45fcd32fd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:53.670720 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67eedf0-af7b-4a07-8f9d-8a45fcd32fd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:53.671225 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67eedf0-af7b-4a07-8f9d-8a45fcd32fd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:53.674620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67eedf0-af7b-4a07-8f9d-8a45fcd32fd0 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:53.688368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 293/559] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:53.689565 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67eedf0-af7b-4a07-8f9d-8a45fcd32fd0 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:53.714536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16447cfd-10dd-4de5-997a-da8194126075 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:53.728321 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:53.763059 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc1c12-1bce-4fe6-8f77-e9aa621e4b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:53.763953 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc1c12-1bce-4fe6-8f77-e9aa621e4b99 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:53.764137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc1c12-1bce-4fe6-8f77-e9aa621e4b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:53.764420 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc1c12-1bce-4fe6-8f77-e9aa621e4b99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:53.784174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc1c12-1bce-4fe6-8f77-e9aa621e4b99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:53.787178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc1c12-1bce-4fe6-8f77-e9aa621e4b99 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:53.814848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-5cfc1c12-1bce-4fe6-8f77-e9aa621e4b99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:53.814848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:53.814848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:53.814848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:53.814848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:53.814848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:53.814848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:53.814848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:53.814848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:53.814848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:53.814848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:53.814848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:53.814848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:53.814848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:53.814848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:53.815893 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:53.815893 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:53.815893 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:53.815893 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:53.815893 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:53.815893 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:53.815893 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:53.815893 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:53.815893 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:53.815893 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:53.815893 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:53.815893 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:53.815893 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:53.815893 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:53.815893 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:53.815893 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:53.815893 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:53.816790 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:53.816790 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:53.816790 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:53.816790 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:53.819205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 267/560] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:53.821049 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc1c12-1bce-4fe6-8f77-e9aa621e4b99 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:53.844437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67eedf0-af7b-4a07-8f9d-8a45fcd32fd0 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:53.907628 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f15e3-355c-4770-b004-685e32d0f7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:53.908320 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f15e3-355c-4770-b004-685e32d0f7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:53.908601 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f15e3-355c-4770-b004-685e32d0f7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:53.908871 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f15e3-355c-4770-b004-685e32d0f7f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:53.909840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f15e3-355c-4770-b004-685e32d0f7f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:53.913125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f15e3-355c-4770-b004-685e32d0f7f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:53.915100 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f15e3-355c-4770-b004-685e32d0f7f6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:53.932819 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 294/561] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:49:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:53.932819 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f15e3-355c-4770-b004-685e32d0f7f6 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:53.958081 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:53.971406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc1c12-1bce-4fe6-8f77-e9aa621e4b99 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:54.047852 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fe89d-67f8-4cd6-8a97-001b605c2d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:54.048964 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fe89d-67f8-4cd6-8a97-001b605c2d07 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:54.049448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fe89d-67f8-4cd6-8a97-001b605c2d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:54.049950 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fe89d-67f8-4cd6-8a97-001b605c2d07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:54.061434 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fe89d-67f8-4cd6-8a97-001b605c2d07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:54.063448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fe89d-67f8-4cd6-8a97-001b605c2d07 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:54.102410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:54.106160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-fe4fe89d-67f8-4cd6-8a97-001b605c2d07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:49:54.106160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:54.106160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:54.106160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:54.106160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:54.106160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:54.106160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:54.106160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:54.106160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:54.106160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:54.106160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:54.106160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:54.106160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:49:54.106160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:49:54.106160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:49:54.107222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:49:54.107222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:54.107222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:54.107222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:54.107222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:54.107222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:54.107222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:54.107222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:54.107222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:54.107222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:54.107222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:54.107222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:54.107222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:54.107222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:54.107222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:54.107222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:54.107222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:54.108107 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:49:54.108107 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:49:54.108107 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:49:54.108107 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:49:54.108107 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:49:54.108107 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:49:54.108107 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:49:54.108107 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:54.115236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 268/562] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:49:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:54.116268 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fe89d-67f8-4cd6-8a97-001b605c2d07 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:54.127298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f15e3-355c-4770-b004-685e32d0f7f6 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:54.190586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3368068-5236-40d6-b4b8-12189f942d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:54.192081 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3368068-5236-40d6-b4b8-12189f942d0d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:54.192823 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3368068-5236-40d6-b4b8-12189f942d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:54.193313 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3368068-5236-40d6-b4b8-12189f942d0d None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:54.194531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3368068-5236-40d6-b4b8-12189f942d0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:54.195191 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3368068-5236-40d6-b4b8-12189f942d0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:54.198307 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3368068-5236-40d6-b4b8-12189f942d0d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:54.219467 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 295/563] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:54.220798 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3368068-5236-40d6-b4b8-12189f942d0d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:54.274562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fe89d-67f8-4cd6-8a97-001b605c2d07 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:54.335044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2fcff-fd8a-4e9c-a072-3359bf2375df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:54.336570 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2fcff-fd8a-4e9c-a072-3359bf2375df None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:54.337578 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2fcff-fd8a-4e9c-a072-3359bf2375df None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:54.337871 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2fcff-fd8a-4e9c-a072-3359bf2375df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:54.354596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2fcff-fd8a-4e9c-a072-3359bf2375df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:54.356709 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2fcff-fd8a-4e9c-a072-3359bf2375df None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:54.372718 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3368068-5236-40d6-b4b8-12189f942d0d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:54.377793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-2ba2fcff-fd8a-4e9c-a072-3359bf2375df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:54.377793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:54.377793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:54.377793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:54.377793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:54.377793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:54.377793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:54.377793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:54.377793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:54.377793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:54.377793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:54.377793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:54.377793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:54.377793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:54.377793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:54.378717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:54.378717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:54.378717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:54.378717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:54.378717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:54.378717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:54.378717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:54.378717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:54.378717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:54.378717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:54.378717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:54.378717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:54.378717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:54.378717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:54.378717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:54.378717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:54.378717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:54.385757 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:54.385757 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:54.385757 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:54.385757 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:54.385757 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 269/564] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:54.385757 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2fcff-fd8a-4e9c-a072-3359bf2375df None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:54.388590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:54.425299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13972abb-8a65-48e8-9da6-582426f6c64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:54.426082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13972abb-8a65-48e8-9da6-582426f6c64c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:54.426428 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13972abb-8a65-48e8-9da6-582426f6c64c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:54.426686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13972abb-8a65-48e8-9da6-582426f6c64c None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:54.427657 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13972abb-8a65-48e8-9da6-582426f6c64c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:54.427971 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13972abb-8a65-48e8-9da6-582426f6c64c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:54.429600 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13972abb-8a65-48e8-9da6-582426f6c64c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:54.453955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 296/565] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:49:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:54.453955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d682ae-ccd3-4967-af51-3c37152c7b1a None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:54.453955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d682ae-ccd3-4967-af51-3c37152c7b1a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:54.453955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d682ae-ccd3-4967-af51-3c37152c7b1a None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:54.453955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 297/566] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:54.453955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d682ae-ccd3-4967-af51-3c37152c7b1a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:54.472099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:54.545685 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2fcff-fd8a-4e9c-a072-3359bf2375df None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:54.584017 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580934b-3c96-46a4-9e2f-a6139b091680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:54.584813 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580934b-3c96-46a4-9e2f-a6139b091680 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:54.585092 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580934b-3c96-46a4-9e2f-a6139b091680 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:54.585358 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580934b-3c96-46a4-9e2f-a6139b091680 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:54.595896 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580934b-3c96-46a4-9e2f-a6139b091680 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:54.597384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9580934b-3c96-46a4-9e2f-a6139b091680 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:54.598013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d682ae-ccd3-4967-af51-3c37152c7b1a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:54.612567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-9580934b-3c96-46a4-9e2f-a6139b091680 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 16 14:49:54.612567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:54.612567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:54.612567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:54.612567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:54.612567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:54.612567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:54.612567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:54.612567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:54.612567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:54.612567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:54.612567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:54.612567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:49:54.612567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:49:54.612567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:49:54.612567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:49:54.613645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:54.613645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:54.613645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:49:54.613645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:49:54.613645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:49:54.613645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:49:54.613645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:54.613645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:54.613645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:54.613645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:54.613645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:54.613645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:54.613645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:54.613645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:54.613645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:54.613645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:54.613645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:54.613645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:54.614568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:54.614568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:54.614568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:49:54.614568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:49:54.614568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:49:54.614568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:49:54.614568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:49:54.614568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:49:54.614568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 16 14:49:54.614568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:54.614568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 270/567] 10.222.0.27 () {56 vars in 1093 bytes} [Sat Jul 16 14:49:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:54.614568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580934b-3c96-46a4-9e2f-a6139b091680 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:54.647685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62b1a3-7b66-40d3-9d35-72a2d5377b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:54.649309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62b1a3-7b66-40d3-9d35-72a2d5377b8d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:54.649664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62b1a3-7b66-40d3-9d35-72a2d5377b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:54.650091 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62b1a3-7b66-40d3-9d35-72a2d5377b8d None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:54.651243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62b1a3-7b66-40d3-9d35-72a2d5377b8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:54.651721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62b1a3-7b66-40d3-9d35-72a2d5377b8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:54.654021 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62b1a3-7b66-40d3-9d35-72a2d5377b8d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:54.671865 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 298/568] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:54.673173 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e194e-83eb-48f9-88fb-63de730b18cb None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:54.673449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e194e-83eb-48f9-88fb-63de730b18cb None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:54.673715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e194e-83eb-48f9-88fb-63de730b18cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:54.693150 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3a8e194e-83eb-48f9-88fb-63de730b18cb None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:54.694356 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3a8e194e-83eb-48f9-88fb-63de730b18cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:54.698950 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:54.743609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580934b-3c96-46a4-9e2f-a6139b091680 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:54.770126 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:54.783388 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08832901-f229-4fbc-bfcb-5654dd079b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:54.783639 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 299/569] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:54.784455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08832901-f229-4fbc-bfcb-5654dd079b0c None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:54.784832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae3eca-c647-4c4b-8ea6-0ebd32b7c8cc None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:54.785113 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08832901-f229-4fbc-bfcb-5654dd079b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:54.785259 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae3eca-c647-4c4b-8ea6-0ebd32b7c8cc None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:54.785387 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08832901-f229-4fbc-bfcb-5654dd079b0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:54.785509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae3eca-c647-4c4b-8ea6-0ebd32b7c8cc None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:54.786050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08832901-f229-4fbc-bfcb-5654dd079b0c None admin] RBAC: Adding query filter params (role.id=21119ebd056740048b9ec639ebb20446, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=e1464c83fb294a3d90e5763ef648e834) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:54.786363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08832901-f229-4fbc-bfcb-5654dd079b0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:54.787009 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 300/570] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:54.787483 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae3eca-c647-4c4b-8ea6-0ebd32b7c8cc None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:54.787906 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08832901-f229-4fbc-bfcb-5654dd079b0c None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:54.794346 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 271/571] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:49:54 2022] GET /identity/v3/role_assignments?user.id=e1464c83fb294a3d90e5763ef648e834&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=21119ebd056740048b9ec639ebb20446 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:54.795305 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08832901-f229-4fbc-bfcb-5654dd079b0c None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:54.975385 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08832901-f229-4fbc-bfcb-5654dd079b0c None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:54.977287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae3eca-c647-4c4b-8ea6-0ebd32b7c8cc None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:55.007177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:55.020446 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4acd61-4697-46ae-8949-fc140f2d38e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:55.021242 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4acd61-4697-46ae-8949-fc140f2d38e1 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:55.021524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4acd61-4697-46ae-8949-fc140f2d38e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:55.022829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4acd61-4697-46ae-8949-fc140f2d38e1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:55.029081 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efe953-9ed5-4302-9150-1458ca8808e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:55.029763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efe953-9ed5-4302-9150-1458ca8808e3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:55.030042 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efe953-9ed5-4302-9150-1458ca8808e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:55.030415 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efe953-9ed5-4302-9150-1458ca8808e3 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:55.031400 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14efe953-9ed5-4302-9150-1458ca8808e3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:55.031711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14efe953-9ed5-4302-9150-1458ca8808e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:55.033167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4acd61-4697-46ae-8949-fc140f2d38e1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:55.033346 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14efe953-9ed5-4302-9150-1458ca8808e3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:55.034629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4acd61-4697-46ae-8949-fc140f2d38e1 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:55.052811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-bc4acd61-4697-46ae-8949-fc140f2d38e1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 16 14:49:55.052811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:55.052811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:55.052811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:55.052811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:55.052811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:55.052811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:55.052811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:55.052811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:55.052811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:55.052811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:55.052811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:55.052811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:49:55.052811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:49:55.052811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:49:55.052811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:49:55.054030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:55.054030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:55.054030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:49:55.054030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:49:55.054030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:49:55.054030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:49:55.054030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:55.054030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:55.054030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:55.054030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:55.054030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:55.054030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:55.054030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:55.054030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:55.054030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:55.054030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:55.054030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:55.054030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:55.060195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:55.060195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:55.060195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:49:55.060195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:49:55.060195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:49:55.060195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:49:55.060195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:49:55.060195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:49:55.060195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 16 14:49:55.060195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:55.060195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 272/572] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:55.060195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12dfd5a-3ebe-4087-997a-b9129dcd93d7 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:55.060195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12dfd5a-3ebe-4087-997a-b9129dcd93d7 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:55.061057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12dfd5a-3ebe-4087-997a-b9129dcd93d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:55.070822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 301/573] 10.222.0.27 () {56 vars in 1133 bytes} [Sat Jul 16 14:49:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:55.070822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efe953-9ed5-4302-9150-1458ca8808e3 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:55.085054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-a12dfd5a-3ebe-4087-997a-b9129dcd93d7 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:55.087152 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a12dfd5a-3ebe-4087-997a-b9129dcd93d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:55.162761 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:55.168217 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 273/574] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:55.169352 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12dfd5a-3ebe-4087-997a-b9129dcd93d7 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:55.238510 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efe953-9ed5-4302-9150-1458ca8808e3 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:55.283642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85218bd-57db-4d5e-98d9-f7a834082ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:55.286554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85218bd-57db-4d5e-98d9-f7a834082ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:55.286554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85218bd-57db-4d5e-98d9-f7a834082ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:55.286554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85218bd-57db-4d5e-98d9-f7a834082ac0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:55.298506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12dfd5a-3ebe-4087-997a-b9129dcd93d7 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:55.300082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85218bd-57db-4d5e-98d9-f7a834082ac0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:55.312633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-e85218bd-57db-4d5e-98d9-f7a834082ac0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:49:55.312633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:55.312633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:55.312633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:55.312633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:55.312633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:55.312633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:55.312633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:55.312633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:55.312633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:55.312633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:55.312633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:55.312633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:55.312633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:55.312633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:55.312633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:55.314405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:55.314405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:55.314405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:55.314405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:55.314405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:55.314405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:55.314405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:55.314405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:55.314405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:55.314405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:55.314405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:55.314405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:55.314405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:55.314405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:55.314405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:55.314405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:55.316080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:55.316080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:55.316080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:55.316080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:55.316080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:55.316080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:55.316080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:55.316080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:55.316080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:55.316080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:55.316080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:49:55.316080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:55.316080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 302/575] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:49:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:55.316080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85218bd-57db-4d5e-98d9-f7a834082ac0 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:55.327700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:55.352652 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc8f48-b516-4d9e-8706-0f507f8a1657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:55.353425 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc8f48-b516-4d9e-8706-0f507f8a1657 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:55.353706 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc8f48-b516-4d9e-8706-0f507f8a1657 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:55.353999 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc8f48-b516-4d9e-8706-0f507f8a1657 None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:55.354826 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc8f48-b516-4d9e-8706-0f507f8a1657 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:55.355167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc8f48-b516-4d9e-8706-0f507f8a1657 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:55.356682 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc8f48-b516-4d9e-8706-0f507f8a1657 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:55.396342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 274/576] 10.222.0.27 () {56 vars in 1137 bytes} [Sat Jul 16 14:49:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:55.397270 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc8f48-b516-4d9e-8706-0f507f8a1657 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:55.450931 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85218bd-57db-4d5e-98d9-f7a834082ac0 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:55.473870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:55.500344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38761f9-becd-4e65-8471-d68943c211bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:55.501634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38761f9-becd-4e65-8471-d68943c211bf None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:55.502069 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38761f9-becd-4e65-8471-d68943c211bf None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:55.502519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38761f9-becd-4e65-8471-d68943c211bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:55.521407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38761f9-becd-4e65-8471-d68943c211bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:55.524360 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38761f9-becd-4e65-8471-d68943c211bf None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:55.548847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc8f48-b516-4d9e-8706-0f507f8a1657 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:55.555748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-b38761f9-becd-4e65-8471-d68943c211bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:55.555748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:55.555748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:55.555748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:55.555748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:55.555748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:55.555748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:55.555748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:55.555748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:55.555748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:55.555748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:55.555748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:55.555748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:55.555748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:55.555748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:55.556893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:55.556893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:55.556893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:55.556893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:55.556893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:55.556893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:55.556893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:55.556893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:55.556893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:55.556893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:55.556893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:55.556893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:55.556893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:55.556893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:55.556893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:55.556893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:55.556893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:55.557945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:55.557945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:55.557945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:55.557945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:55.564532 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 303/577] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:55.564532 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38761f9-becd-4e65-8471-d68943c211bf None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:55.599329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88596d75-36fc-49ea-bcf1-a7fb62a78e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:55.600130 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88596d75-36fc-49ea-bcf1-a7fb62a78e74 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:55.600422 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88596d75-36fc-49ea-bcf1-a7fb62a78e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:55.600696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88596d75-36fc-49ea-bcf1-a7fb62a78e74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:55.618321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88596d75-36fc-49ea-bcf1-a7fb62a78e74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:55.626132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-88596d75-36fc-49ea-bcf1-a7fb62a78e74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:49:55.626132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:55.626132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:55.626132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:55.626132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:55.626132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:55.626132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:55.626132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:55.626132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:55.626132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:55.626132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:55.626132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:55.626132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:55.626132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:55.626132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:55.626132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:55.627127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:55.627127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:55.627127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:55.627127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:55.627127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:55.627127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:55.627127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:55.627127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:55.627127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:55.627127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:55.627127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:55.627127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:55.627127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:55.627127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:55.627127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:55.627127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:55.628002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:55.628002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:55.628002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:55.628002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:55.628002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:55.628002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:55.628002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:55.628002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:55.628002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:55.628002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:55.628002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:49:55.628002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:55.628002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 275/578] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:49:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:55.629515 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88596d75-36fc-49ea-bcf1-a7fb62a78e74 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:55.671447 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:55.696929 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38761f9-becd-4e65-8471-d68943c211bf None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:55.740720 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cdc84b-fef9-4050-b642-6c2acf40a884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:55.741399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdc84b-fef9-4050-b642-6c2acf40a884 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:55.741687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdc84b-fef9-4050-b642-6c2acf40a884 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:55.742540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdc84b-fef9-4050-b642-6c2acf40a884 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:55.743271 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cdc84b-fef9-4050-b642-6c2acf40a884 None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:55.743808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cdc84b-fef9-4050-b642-6c2acf40a884 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:55.745314 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cdc84b-fef9-4050-b642-6c2acf40a884 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:55.754770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 304/579] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:49:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:55.755511 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cdc84b-fef9-4050-b642-6c2acf40a884 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:55.761666 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88596d75-36fc-49ea-bcf1-a7fb62a78e74 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:55.805217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:55.808464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6722262-9827-4153-9370-eb528a1d49f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:55.809335 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6722262-9827-4153-9370-eb528a1d49f9 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:55.809616 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6722262-9827-4153-9370-eb528a1d49f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:55.809907 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6722262-9827-4153-9370-eb528a1d49f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:55.821624 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6722262-9827-4153-9370-eb528a1d49f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:55.823134 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6722262-9827-4153-9370-eb528a1d49f9 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:55.847183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-f6722262-9827-4153-9370-eb528a1d49f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:55.847183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:55.847183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:55.847183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:55.847183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:55.847183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:55.847183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:55.847183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:55.847183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:55.847183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:55.847183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:55.847183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:55.847183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:55.847183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:55.847183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:55.848150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:55.848150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:55.848150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:55.848150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:55.848150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:55.848150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:55.848150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:55.848150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:55.848150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:55.848150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:55.848150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:55.848150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:55.848150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:55.848150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:55.848150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:55.848150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:55.848150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:55.849027 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:55.849027 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:55.849027 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:55.849027 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:55.849027 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 276/580] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:55.849324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6722262-9827-4153-9370-eb528a1d49f9 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:55.915919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cdc84b-fef9-4050-b642-6c2acf40a884 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:55.974648 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0e243-a0af-424d-a6c9-cdcb540ca178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:55.975481 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0e243-a0af-424d-a6c9-cdcb540ca178 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:55.975777 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0e243-a0af-424d-a6c9-cdcb540ca178 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:55.976058 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0e243-a0af-424d-a6c9-cdcb540ca178 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:55.976913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b0e243-a0af-424d-a6c9-cdcb540ca178 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:55.977224 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b0e243-a0af-424d-a6c9-cdcb540ca178 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:55.978652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b0e243-a0af-424d-a6c9-cdcb540ca178 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:55.980221 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:55.993602 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 305/581] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:55.994502 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0e243-a0af-424d-a6c9-cdcb540ca178 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:56.001185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6722262-9827-4153-9370-eb528a1d49f9 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:56.042545 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10678a-037f-4d69-833d-35f491d1607b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:56.043312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10678a-037f-4d69-833d-35f491d1607b None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:56.043763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10678a-037f-4d69-833d-35f491d1607b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:56.044043 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10678a-037f-4d69-833d-35f491d1607b None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:56.044827 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10678a-037f-4d69-833d-35f491d1607b None admin] RBAC: Adding query filter params (name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:56.045146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10678a-037f-4d69-833d-35f491d1607b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:56.046596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c10678a-037f-4d69-833d-35f491d1607b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:56.056623 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 277/582] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:49:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:56.057430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10678a-037f-4d69-833d-35f491d1607b None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:56.120622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:56.135675 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0e243-a0af-424d-a6c9-cdcb540ca178 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:56.192139 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40863fd1-92ea-441f-9df5-b09950c0dc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:56.193109 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40863fd1-92ea-441f-9df5-b09950c0dc3d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:56.193509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40863fd1-92ea-441f-9df5-b09950c0dc3d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:56.193963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40863fd1-92ea-441f-9df5-b09950c0dc3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:56.197273 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10678a-037f-4d69-833d-35f491d1607b None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:56.207942 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40863fd1-92ea-441f-9df5-b09950c0dc3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:56.209848 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40863fd1-92ea-441f-9df5-b09950c0dc3d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:56.243269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-40863fd1-92ea-441f-9df5-b09950c0dc3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:56.243269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:56.243269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:56.243269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:56.243269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:56.243269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:56.243269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:56.243269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:56.243269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:56.243269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:56.243269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:56.243269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:56.243269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:56.243269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:56.243269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:56.244231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:56.244231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:56.244231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:56.244231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:56.244231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:56.244231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:56.244231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:56.244231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:56.244231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:56.244231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:56.244231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:56.244231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:56.244231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:56.244231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:56.244231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:56.244231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:56.244231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:56.245102 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:56.245102 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:56.245102 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:56.245102 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:56.245102 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 306/583] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:56.246329 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40863fd1-92ea-441f-9df5-b09950c0dc3d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:56.256532 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d92747-48e6-421c-8506-962424ddea5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:56.257325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d92747-48e6-421c-8506-962424ddea5a None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:56.257638 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d92747-48e6-421c-8506-962424ddea5a None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:56.257923 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d92747-48e6-421c-8506-962424ddea5a None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:56.258655 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d92747-48e6-421c-8506-962424ddea5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:56.258984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d92747-48e6-421c-8506-962424ddea5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:56.260378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d92747-48e6-421c-8506-962424ddea5a None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:56.275378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 278/584] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:56.276337 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d92747-48e6-421c-8506-962424ddea5a None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:56.290382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:56.389545 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40863fd1-92ea-441f-9df5-b09950c0dc3d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:56.410485 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d92747-48e6-421c-8506-962424ddea5a None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:56.436608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cacc3-a951-4778-95e4-dc1641d2bd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:56.438048 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cacc3-a951-4778-95e4-dc1641d2bd23 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:56.438675 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cacc3-a951-4778-95e4-dc1641d2bd23 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:56.439167 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cacc3-a951-4778-95e4-dc1641d2bd23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:56.448719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452cacc3-a951-4778-95e4-dc1641d2bd23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:56.450300 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452cacc3-a951-4778-95e4-dc1641d2bd23 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:56.457050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b11e0-86a2-4ecf-8c51-2a7f5019ac63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:56.458033 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b11e0-86a2-4ecf-8c51-2a7f5019ac63 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:56.458450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b11e0-86a2-4ecf-8c51-2a7f5019ac63 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:56.458850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b11e0-86a2-4ecf-8c51-2a7f5019ac63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:56.466524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-452cacc3-a951-4778-95e4-dc1641d2bd23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:49:56.466524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:56.466524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:56.466524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:56.466524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:56.466524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:56.466524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:56.466524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:56.466524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:56.466524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:56.466524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:56.466524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:56.466524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:49:56.466524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:49:56.466524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:49:56.467826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:49:56.467826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:56.467826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:56.467826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:56.467826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:56.467826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:56.467826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:56.467826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:56.467826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:56.467826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:56.467826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:56.467826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:56.467826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:56.467826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:56.467826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:56.467826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:56.467826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:56.468961 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:49:56.468961 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:49:56.468961 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:49:56.468961 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:49:56.468961 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:49:56.468961 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:49:56.468961 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:49:56.468961 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:56.468961 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 307/585] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:49:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:56.468961 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cacc3-a951-4778-95e4-dc1641d2bd23 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:56.473862 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:56.477483 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b11e0-86a2-4ecf-8c51-2a7f5019ac63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:56.479060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b11e0-86a2-4ecf-8c51-2a7f5019ac63 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:56.500317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-d42b11e0-86a2-4ecf-8c51-2a7f5019ac63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:56.500317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:56.500317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:56.500317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:56.500317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:56.500317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:56.500317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:56.500317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:56.500317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:56.500317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:56.500317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:56.500317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:56.500317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:56.500317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:56.500317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:56.501963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:56.501963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:56.501963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:56.501963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:56.501963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:56.501963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:56.501963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:56.501963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:56.501963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:56.501963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:56.501963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:56.501963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:56.501963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:56.501963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:56.501963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:56.501963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:56.501963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:56.504279 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:56.504279 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:56.504279 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:56.504279 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:56.504279 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 279/586] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:56.504279 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b11e0-86a2-4ecf-8c51-2a7f5019ac63 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:56.574588 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:56.618583 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cacc3-a951-4778-95e4-dc1641d2bd23 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:56.627538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b11e0-86a2-4ecf-8c51-2a7f5019ac63 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:56.672276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399879a-41db-4f6b-90ac-e7692d14a702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:56.672472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc9163-4a89-4377-86db-b271abda5b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:56.673211 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc9163-4a89-4377-86db-b271abda5b73 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:56.673424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399879a-41db-4f6b-90ac-e7692d14a702 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:56.673668 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc9163-4a89-4377-86db-b271abda5b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:56.673879 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399879a-41db-4f6b-90ac-e7692d14a702 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:56.674120 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc9163-4a89-4377-86db-b271abda5b73 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:56.674325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399879a-41db-4f6b-90ac-e7692d14a702 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:56.675404 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc9163-4a89-4377-86db-b271abda5b73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:56.675898 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc9163-4a89-4377-86db-b271abda5b73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:56.678163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc9163-4a89-4377-86db-b271abda5b73 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:56.687269 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399879a-41db-4f6b-90ac-e7692d14a702 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:56.690783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399879a-41db-4f6b-90ac-e7692d14a702 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:56.703692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 308/587] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:56.703692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc9163-4a89-4377-86db-b271abda5b73 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:56.719856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-7399879a-41db-4f6b-90ac-e7692d14a702 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:49:56.719856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:56.719856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:56.719856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:56.719856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:56.719856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:56.719856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:56.719856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:56.719856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:56.719856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:56.719856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:56.719856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:56.719856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:49:56.719856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:49:56.719856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:49:56.720811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:49:56.720811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:56.720811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:56.720811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:56.720811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:56.720811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:56.720811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:56.720811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:56.720811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:56.720811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:56.720811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:56.720811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:56.720811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:56.720811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:56.720811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:56.720811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:56.720811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:56.721694 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:49:56.721694 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:49:56.721694 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:49:56.721694 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:49:56.721694 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:49:56.721694 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:49:56.721694 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:49:56.721694 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:56.727871 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 280/588] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:49:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:56.727871 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399879a-41db-4f6b-90ac-e7692d14a702 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:56.790121 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:56.838216 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc9163-4a89-4377-86db-b271abda5b73 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:56.881870 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399879a-41db-4f6b-90ac-e7692d14a702 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:56.882158 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff67a2-cb20-4bb0-a2c8-cee83cd7323e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:56.883159 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff67a2-cb20-4bb0-a2c8-cee83cd7323e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:56.884244 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff67a2-cb20-4bb0-a2c8-cee83cd7323e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:56.884645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff67a2-cb20-4bb0-a2c8-cee83cd7323e None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:56.886543 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ff67a2-cb20-4bb0-a2c8-cee83cd7323e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:56.886853 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ff67a2-cb20-4bb0-a2c8-cee83cd7323e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:56.888399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ff67a2-cb20-4bb0-a2c8-cee83cd7323e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:56.902795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 309/589] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:49:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:56.904000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff67a2-cb20-4bb0-a2c8-cee83cd7323e None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:56.913082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:56.930709 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3968d72-bf47-4379-a56e-96bea0236278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:56.931728 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3968d72-bf47-4379-a56e-96bea0236278 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:56.932006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3968d72-bf47-4379-a56e-96bea0236278 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:56.932274 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3968d72-bf47-4379-a56e-96bea0236278 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:56.933116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3968d72-bf47-4379-a56e-96bea0236278 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:56.933392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3968d72-bf47-4379-a56e-96bea0236278 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:56.934793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3968d72-bf47-4379-a56e-96bea0236278 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:56.950835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 281/590] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:56.952434 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3968d72-bf47-4379-a56e-96bea0236278 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:57.025687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff67a2-cb20-4bb0-a2c8-cee83cd7323e None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:57.058470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:57.075985 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846d9a3-eed5-49dc-b34e-cc485008f724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:57.077202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846d9a3-eed5-49dc-b34e-cc485008f724 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:57.077665 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846d9a3-eed5-49dc-b34e-cc485008f724 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:57.078119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846d9a3-eed5-49dc-b34e-cc485008f724 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:57.092908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7846d9a3-eed5-49dc-b34e-cc485008f724 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:57.095310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7846d9a3-eed5-49dc-b34e-cc485008f724 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:57.106105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3968d72-bf47-4379-a56e-96bea0236278 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:57.112733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-7846d9a3-eed5-49dc-b34e-cc485008f724 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 16 14:49:57.112733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:57.112733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:57.112733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:57.112733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:57.112733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:57.112733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:57.112733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:57.112733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:57.112733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:57.112733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:57.112733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:57.112733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:49:57.112733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:49:57.112733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:49:57.112733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:49:57.114780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:57.114780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:57.114780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:49:57.114780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:49:57.114780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:49:57.114780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:49:57.114780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:57.114780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:57.114780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:57.114780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:57.114780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:57.114780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:57.114780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:57.114780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:57.114780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:57.114780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:57.114780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:57.114780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:57.116696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:57.116696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:57.116696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:49:57.116696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:49:57.116696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:49:57.116696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:49:57.116696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:49:57.116696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:49:57.116696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 16 14:49:57.116696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:57.116696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 310/591] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:49:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:57.116696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36f4bb-793e-45ba-9aaa-1f464927283e None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:57.116696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36f4bb-793e-45ba-9aaa-1f464927283e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:57.118349 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36f4bb-793e-45ba-9aaa-1f464927283e None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:57.118349 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 311/592] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:49:57.119258 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36f4bb-793e-45ba-9aaa-1f464927283e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:57.164913 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4483dc-f0e9-469d-ba1c-4200104c4122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:57.165587 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4483dc-f0e9-469d-ba1c-4200104c4122 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:57.165862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4483dc-f0e9-469d-ba1c-4200104c4122 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:57.166132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4483dc-f0e9-469d-ba1c-4200104c4122 None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:57.166951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4483dc-f0e9-469d-ba1c-4200104c4122 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:57.167258 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4483dc-f0e9-469d-ba1c-4200104c4122 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:57.168712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4483dc-f0e9-469d-ba1c-4200104c4122 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:57.183214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 282/593] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:49:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:57.185090 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4483dc-f0e9-469d-ba1c-4200104c4122 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:57.233495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:57.237552 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36f4bb-793e-45ba-9aaa-1f464927283e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:57.279742 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e75d98-1ed4-4193-a9c9-82afc04562c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:57.281432 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e75d98-1ed4-4193-a9c9-82afc04562c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:57.281884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e75d98-1ed4-4193-a9c9-82afc04562c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:57.282277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e75d98-1ed4-4193-a9c9-82afc04562c3 None admin] PATH_INFO: `/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/3f2e241494244edf8d283e8f44192c46/roles/21119ebd056740048b9ec639ebb20446` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:57.310226 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e75d98-1ed4-4193-a9c9-82afc04562c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=2739f00eb01e46a9acf90ffa360d7d8c, user_id=3f2e241494244edf8d283e8f44192c46, role_id=21119ebd056740048b9ec639ebb20446)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:57.312829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e75d98-1ed4-4193-a9c9-82afc04562c3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:57.319384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4483dc-f0e9-469d-ba1c-4200104c4122 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:57.363577 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669761f-9c41-410d-af24-984d0c86a8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:57.364401 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669761f-9c41-410d-af24-984d0c86a8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:57.364679 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669761f-9c41-410d-af24-984d0c86a8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:57.364948 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669761f-9c41-410d-af24-984d0c86a8c8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:57.374680 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669761f-9c41-410d-af24-984d0c86a8c8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:57.376403 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669761f-9c41-410d-af24-984d0c86a8c8 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:57.388855 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 312/594] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:49:57 2022] PUT /identity/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/3f2e241494244edf8d283e8f44192c46/roles/21119ebd056740048b9ec639ebb20446 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:49:57.390395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e75d98-1ed4-4193-a9c9-82afc04562c3 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:57.397911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-6669761f-9c41-410d-af24-984d0c86a8c8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 16 14:49:57.397911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:57.397911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:57.397911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:57.397911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:57.397911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:57.397911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:57.397911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:57.397911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:57.397911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:57.397911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:57.397911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:57.397911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:49:57.397911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:49:57.397911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:49:57.397911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:49:57.409632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:57.409632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:57.409632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:49:57.409632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:49:57.409632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:49:57.409632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:49:57.409632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:57.409632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:57.409632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:57.409632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:57.409632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:57.409632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:57.409632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:57.409632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:57.409632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:57.409632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:57.409632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:57.409632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:57.411311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:57.411311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:57.411311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:49:57.411311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:49:57.411311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:49:57.411311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:49:57.411311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:49:57.411311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:49:57.411311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 16 14:49:57.411311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:57.411311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 283/595] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:49:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:57.411311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226831c-89ec-4d66-8fa4-0e37253a0b4a None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:57.411311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226831c-89ec-4d66-8fa4-0e37253a0b4a None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:57.412642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226831c-89ec-4d66-8fa4-0e37253a0b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:57.432776 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-3226831c-89ec-4d66-8fa4-0e37253a0b4a None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:49:57.434664 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3226831c-89ec-4d66-8fa4-0e37253a0b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:49:57.475864 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:57.585118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 284/596] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:49:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:49:57.586722 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e75d98-1ed4-4193-a9c9-82afc04562c3 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:57.588348 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226831c-89ec-4d66-8fa4-0e37253a0b4a None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:57.628081 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:57.652933 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df39b6f-8034-4cc4-9165-463a143c5d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:57.654122 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df39b6f-8034-4cc4-9165-463a143c5d4b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:57.654624 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df39b6f-8034-4cc4-9165-463a143c5d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:57.655117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df39b6f-8034-4cc4-9165-463a143c5d4b None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:57.656834 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df39b6f-8034-4cc4-9165-463a143c5d4b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:57.657333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df39b6f-8034-4cc4-9165-463a143c5d4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:57.660061 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df39b6f-8034-4cc4-9165-463a143c5d4b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:57.699986 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 313/597] 10.222.0.27 () {56 vars in 1137 bytes} [Sat Jul 16 14:49:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:57.701424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df39b6f-8034-4cc4-9165-463a143c5d4b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:57.710619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226831c-89ec-4d66-8fa4-0e37253a0b4a None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:57.776618 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0050c0-d3e2-4700-966a-c7407ffc1704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:57.778011 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0050c0-d3e2-4700-966a-c7407ffc1704 None admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:57.778537 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0050c0-d3e2-4700-966a-c7407ffc1704 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:57.779042 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0050c0-d3e2-4700-966a-c7407ffc1704 None admin] PATH_INFO: `/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/7d689162291f4f4b97e886ea20d3a8b2/roles/21119ebd056740048b9ec639ebb20446` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:57.805727 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0050c0-d3e2-4700-966a-c7407ffc1704 None admin] RBAC: Authorizing `identity:create_grant(project_id=2739f00eb01e46a9acf90ffa360d7d8c, user_id=7d689162291f4f4b97e886ea20d3a8b2, role_id=21119ebd056740048b9ec639ebb20446)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:57.808240 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0050c0-d3e2-4700-966a-c7407ffc1704 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:57.844973 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:57.856416 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df39b6f-8034-4cc4-9165-463a143c5d4b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:57.893586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a348e5-6bd4-458a-85c7-5999822f014c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:57.894260 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a348e5-6bd4-458a-85c7-5999822f014c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:57.894541 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a348e5-6bd4-458a-85c7-5999822f014c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:57.894812 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a348e5-6bd4-458a-85c7-5999822f014c None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:57.895687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a348e5-6bd4-458a-85c7-5999822f014c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:57.895999 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a348e5-6bd4-458a-85c7-5999822f014c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:57.897470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a348e5-6bd4-458a-85c7-5999822f014c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:57.914932 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 285/598] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:49:57 2022] PUT /identity/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/7d689162291f4f4b97e886ea20d3a8b2/roles/21119ebd056740048b9ec639ebb20446 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:49:57.916236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0050c0-d3e2-4700-966a-c7407ffc1704 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:57.930985 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 314/599] 10.222.0.27 () {56 vars in 1139 bytes} [Sat Jul 16 14:49:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:57.932349 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a348e5-6bd4-458a-85c7-5999822f014c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:58.111444 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:58.146312 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a348e5-6bd4-458a-85c7-5999822f014c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:58.167621 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0050c0-d3e2-4700-966a-c7407ffc1704 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:58.224718 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f160a3-21dc-4d64-980c-28f63fe0798f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:58.227643 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f160a3-21dc-4d64-980c-28f63fe0798f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:58.227643 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f160a3-21dc-4d64-980c-28f63fe0798f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:58.228887 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f160a3-21dc-4d64-980c-28f63fe0798f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:58.244499 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6810f063-0486-456b-9780-2b0f234dedb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:58.245342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810f063-0486-456b-9780-2b0f234dedb1 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:58.245634 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810f063-0486-456b-9780-2b0f234dedb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:58.246052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810f063-0486-456b-9780-2b0f234dedb1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:58.247951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f160a3-21dc-4d64-980c-28f63fe0798f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:58.251123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f160a3-21dc-4d64-980c-28f63fe0798f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:58.259549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6810f063-0486-456b-9780-2b0f234dedb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:49:58.268062 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:58.269592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-6810f063-0486-456b-9780-2b0f234dedb1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:49:58.269592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:58.269592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:58.269592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:58.269592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:58.269592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:58.269592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:58.269592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:58.269592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:58.269592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:58.269592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:58.269592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:58.269592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:49:58.269592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:49:58.269592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:49:58.269592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:49:58.271524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:49:58.271524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:49:58.271524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:49:58.271524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:49:58.271524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:58.271524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:58.271524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:58.271524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:58.271524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:58.271524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:58.271524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:58.271524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:58.271524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:58.271524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:58.271524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:58.271524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:58.273209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:58.273209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:58.273209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:58.273209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:58.273209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:49:58.273209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:49:58.273209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:49:58.273209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:49:58.273209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:49:58.273209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:49:58.273209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:49:58.273209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:58.273209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 286/600] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:49:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:58.273209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6810f063-0486-456b-9780-2b0f234dedb1 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:58.296441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-a9f160a3-21dc-4d64-980c-28f63fe0798f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:58.296441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:58.296441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:58.296441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:58.296441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:58.296441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:58.296441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:58.296441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:58.296441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:58.296441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:58.296441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:58.296441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:58.296441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:58.296441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:58.296441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:58.299691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:58.299691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:58.299691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:58.299691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:58.299691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:58.299691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:58.299691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:58.299691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:58.299691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:58.299691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:58.299691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:58.299691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:58.299691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:58.299691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:58.299691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:58.299691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:58.299691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:58.303171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:58.303171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:58.303171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:58.303171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:58.303171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 315/601] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:58.303171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f160a3-21dc-4d64-980c-28f63fe0798f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:58.409218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6810f063-0486-456b-9780-2b0f234dedb1 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:58.458359 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81fe529-4cf7-4349-9772-69c8caef7669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:58.459527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fe529-4cf7-4349-9772-69c8caef7669 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:58.459810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fe529-4cf7-4349-9772-69c8caef7669 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:58.460077 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fe529-4cf7-4349-9772-69c8caef7669 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:58.473260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:58.476674 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81fe529-4cf7-4349-9772-69c8caef7669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:58.478816 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81fe529-4cf7-4349-9772-69c8caef7669 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:58.498474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f160a3-21dc-4d64-980c-28f63fe0798f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:58.500985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-b81fe529-4cf7-4349-9772-69c8caef7669 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:58.500985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:58.500985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:58.500985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:58.500985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:58.500985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:58.500985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:58.500985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:58.500985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:58.500985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:58.500985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:58.500985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:58.500985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:58.500985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:58.500985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:58.502607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:58.502607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:58.502607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:58.502607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:58.502607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:58.502607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:58.502607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:58.502607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:58.502607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:58.502607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:58.502607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:58.502607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:58.502607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:58.502607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:58.502607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:58.502607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:58.502607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:58.504343 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:58.504343 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:58.504343 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:58.504343 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:58.504343 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 287/602] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:58.509831 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81fe529-4cf7-4349-9772-69c8caef7669 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:58.566186 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1684fa-168d-4958-98a3-432a7dfa048c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:58.567464 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1684fa-168d-4958-98a3-432a7dfa048c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:58.567964 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1684fa-168d-4958-98a3-432a7dfa048c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:58.568455 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1684fa-168d-4958-98a3-432a7dfa048c None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:58.570673 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1684fa-168d-4958-98a3-432a7dfa048c None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:58.571264 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1684fa-168d-4958-98a3-432a7dfa048c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:58.574038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1684fa-168d-4958-98a3-432a7dfa048c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:58.598072 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 316/603] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:49:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:58.599768 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1684fa-168d-4958-98a3-432a7dfa048c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:58.654640 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81fe529-4cf7-4349-9772-69c8caef7669 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:58.700482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:58.722087 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05bcc26-bfc8-4a2c-a345-8581861146db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:58.723380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bcc26-bfc8-4a2c-a345-8581861146db None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:58.723883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bcc26-bfc8-4a2c-a345-8581861146db None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:58.724367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bcc26-bfc8-4a2c-a345-8581861146db None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:58.725757 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05bcc26-bfc8-4a2c-a345-8581861146db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:58.726448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05bcc26-bfc8-4a2c-a345-8581861146db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:58.729621 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05bcc26-bfc8-4a2c-a345-8581861146db None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:58.763084 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 288/604] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:58.764885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05bcc26-bfc8-4a2c-a345-8581861146db None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:58.776026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1684fa-168d-4958-98a3-432a7dfa048c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:58.864254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3dbee-c786-46f2-a299-09fb74f7d8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:58.865113 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3dbee-c786-46f2-a299-09fb74f7d8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:58.865555 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3dbee-c786-46f2-a299-09fb74f7d8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:58.865891 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3dbee-c786-46f2-a299-09fb74f7d8a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:58.866980 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3dbee-c786-46f2-a299-09fb74f7d8a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:58.867554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3dbee-c786-46f2-a299-09fb74f7d8a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:58.869163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3dbee-c786-46f2-a299-09fb74f7d8a6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:58.882942 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 317/605] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:58.883441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3dbee-c786-46f2-a299-09fb74f7d8a6 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:58.923112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:59.002822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3dbee-c786-46f2-a299-09fb74f7d8a6 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:59.018248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05bcc26-bfc8-4a2c-a345-8581861146db None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:59.032180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:59.045831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b12ac9-e87b-466c-ab71-7767efb52705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:59.047098 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b12ac9-e87b-466c-ab71-7767efb52705 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:59.047390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b12ac9-e87b-466c-ab71-7767efb52705 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:59.047809 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b12ac9-e87b-466c-ab71-7767efb52705 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:59.056689 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b12ac9-e87b-466c-ab71-7767efb52705 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:59.058537 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b12ac9-e87b-466c-ab71-7767efb52705 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:59.076463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-98b12ac9-e87b-466c-ab71-7767efb52705 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:49:59.076463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:59.076463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:59.076463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:59.076463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:59.076463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:59.076463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:59.076463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:59.076463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:59.076463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:59.076463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:59.076463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:59.076463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:49:59.076463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:49:59.076463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:49:59.077424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:49:59.077424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:59.077424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:59.077424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:59.077424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:59.077424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:59.077424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:59.077424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:59.077424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:59.077424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:59.077424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:59.077424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:59.077424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:59.077424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:59.077424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:59.077424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:59.077424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:59.078303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:49:59.078303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:49:59.078303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:49:59.078303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:49:59.078303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:49:59.078303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:49:59.078303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:49:59.078303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:59.081104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 318/606] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:49:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:59.082241 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b12ac9-e87b-466c-ab71-7767efb52705 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:59.100369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550f6e6-b172-4140-a5d3-e7493dcbfeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:59.102131 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550f6e6-b172-4140-a5d3-e7493dcbfeb1 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:59.102630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550f6e6-b172-4140-a5d3-e7493dcbfeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:59.103182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550f6e6-b172-4140-a5d3-e7493dcbfeb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:59.126501 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550f6e6-b172-4140-a5d3-e7493dcbfeb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:59.129196 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550f6e6-b172-4140-a5d3-e7493dcbfeb1 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:59.165048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-7550f6e6-b172-4140-a5d3-e7493dcbfeb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:59.165048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:59.165048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:59.165048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:59.165048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:59.165048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:59.165048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:59.165048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:59.165048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:59.165048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:59.165048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:59.165048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:59.165048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:49:59.165048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:49:59.165048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:49:59.166529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:49:59.166529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:59.166529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:59.166529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:59.166529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:59.166529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:59.166529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:59.166529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:59.166529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:59.166529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:59.166529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:59.166529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:59.166529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:59.166529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:59.166529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:59.166529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:59.166529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:59.167884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:49:59.167884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:49:59.167884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:49:59.167884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:49:59.167884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 289/607] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:49:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:59.169005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550f6e6-b172-4140-a5d3-e7493dcbfeb1 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:59.293531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b12ac9-e87b-466c-ab71-7767efb52705 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:59.358431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6825d-7920-42a1-930a-4903e5e04965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:59.360260 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6825d-7920-42a1-930a-4903e5e04965 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:59.361024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6825d-7920-42a1-930a-4903e5e04965 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:59.361808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6825d-7920-42a1-930a-4903e5e04965 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:59.375121 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6825d-7920-42a1-930a-4903e5e04965 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:59.376350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550f6e6-b172-4140-a5d3-e7493dcbfeb1 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:59.378384 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6825d-7920-42a1-930a-4903e5e04965 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:59.381166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:59.407858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-0ec6825d-7920-42a1-930a-4903e5e04965 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:49:59.407858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:49:59.407858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:49:59.407858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:49:59.407858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:49:59.407858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:49:59.407858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:49:59.407858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:49:59.407858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:49:59.407858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:49:59.407858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:49:59.407858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:49:59.407858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:49:59.407858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:49:59.407858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:49:59.409441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:49:59.409441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:49:59.409441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:49:59.409441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:49:59.409441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:49:59.409441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:49:59.409441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:49:59.409441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:49:59.409441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:49:59.409441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:49:59.409441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:49:59.409441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:49:59.409441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:49:59.409441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:49:59.409441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:49:59.409441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:49:59.409441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:49:59.411147 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:49:59.411147 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:49:59.411147 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:49:59.411147 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:49:59.411147 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:49:59.411147 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:49:59.411147 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:49:59.411147 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:49:59.415938 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 319/608] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:49:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:49:59.417847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6825d-7920-42a1-930a-4903e5e04965 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:59.461157 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830a098-0dc8-45dc-8196-89ad33237d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:59.463039 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830a098-0dc8-45dc-8196-89ad33237d79 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:59.463841 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830a098-0dc8-45dc-8196-89ad33237d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:59.464587 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830a098-0dc8-45dc-8196-89ad33237d79 None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:59.467077 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830a098-0dc8-45dc-8196-89ad33237d79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:59.467716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830a098-0dc8-45dc-8196-89ad33237d79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:59.470509 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830a098-0dc8-45dc-8196-89ad33237d79 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:59.503349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 290/609] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:49:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:59.505127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830a098-0dc8-45dc-8196-89ad33237d79 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:59.519313 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:49:59.653859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6825d-7920-42a1-930a-4903e5e04965 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:59.738256 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0f441-8c76-4d2d-a9e9-777998d2d39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:59.739006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830a098-0dc8-45dc-8196-89ad33237d79 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:49:59.739681 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0f441-8c76-4d2d-a9e9-777998d2d39d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:59.740150 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0f441-8c76-4d2d-a9e9-777998d2d39d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:59.740616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0f441-8c76-4d2d-a9e9-777998d2d39d None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:59.742069 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd0f441-8c76-4d2d-a9e9-777998d2d39d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:59.742739 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd0f441-8c76-4d2d-a9e9-777998d2d39d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:59.745744 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd0f441-8c76-4d2d-a9e9-777998d2d39d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:59.781251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 320/610] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:49:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:59.783720 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0f441-8c76-4d2d-a9e9-777998d2d39d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:59.823503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540372db-416c-4c24-9352-53fb4047fc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:49:59.825376 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540372db-416c-4c24-9352-53fb4047fc90 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:49:59.825955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540372db-416c-4c24-9352-53fb4047fc90 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:49:59.826499 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540372db-416c-4c24-9352-53fb4047fc90 None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:49:59.828427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540372db-416c-4c24-9352-53fb4047fc90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:49:59.829115 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540372db-416c-4c24-9352-53fb4047fc90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:49:59.832541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540372db-416c-4c24-9352-53fb4047fc90 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:49:59.870556 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 291/611] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:49:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:49:59.870556 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540372db-416c-4c24-9352-53fb4047fc90 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:49:59.941301 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:00.041452 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0f441-8c76-4d2d-a9e9-777998d2d39d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:00.046015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:00.113801 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9a868-e86e-45ed-a456-40140b245b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:00.115280 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540372db-416c-4c24-9352-53fb4047fc90 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:00.116267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9a868-e86e-45ed-a456-40140b245b89 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:00.117614 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9a868-e86e-45ed-a456-40140b245b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:00.118422 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9a868-e86e-45ed-a456-40140b245b89 None admin] PATH_INFO: `/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/8c21d12c977e4c36b4f0ed442b11ef49/roles/21119ebd056740048b9ec639ebb20446` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:00.149325 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9a868-e86e-45ed-a456-40140b245b89 None admin] RBAC: Authorizing `identity:create_grant(project_id=2739f00eb01e46a9acf90ffa360d7d8c, user_id=8c21d12c977e4c36b4f0ed442b11ef49, role_id=21119ebd056740048b9ec639ebb20446)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:00.152608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9a868-e86e-45ed-a456-40140b245b89 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:00.174567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db298673-aecc-4903-aace-a3321d7b47ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:00.175852 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db298673-aecc-4903-aace-a3321d7b47ae None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:00.176358 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db298673-aecc-4903-aace-a3321d7b47ae None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:00.176844 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db298673-aecc-4903-aace-a3321d7b47ae None admin] PATH_INFO: `/v3/users/swift` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:00.191203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db298673-aecc-4903-aace-a3321d7b47ae None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:00.193958 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db298673-aecc-4903-aace-a3321d7b47ae None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:00.223924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-db298673-aecc-4903-aace-a3321d7b47ae None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 16 14:50:00.223924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:00.223924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:00.223924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:00.223924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:00.223924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:00.223924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:00.223924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:00.223924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:00.223924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:00.223924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:00.223924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:00.223924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:50:00.223924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:50:00.223924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:50:00.223924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:50:00.225060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:00.225060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:00.225060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:50:00.225060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:00.225060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:50:00.225060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:00.225060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:00.225060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:00.225060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:00.225060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:00.225060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:00.225060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:00.225060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:00.225060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:00.225060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:00.225060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:00.225060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:00.225060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:00.226205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:00.226205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:00.226205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:50:00.226205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:50:00.226205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:50:00.226205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:50:00.226205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:50:00.226205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:50:00.226205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 16 14:50:00.226205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:00.227760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 292/612] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:49:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:00.228214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db298673-aecc-4903-aace-a3321d7b47ae None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:00.242944 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 321/613] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:49:59 2022] PUT /identity/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/8c21d12c977e4c36b4f0ed442b11ef49/roles/21119ebd056740048b9ec639ebb20446 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:50:00.245704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8c3c3-04b9-458f-abf4-e8157cec415d None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:00.246474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8c3c3-04b9-458f-abf4-e8157cec415d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:00.247228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8c3c3-04b9-458f-abf4-e8157cec415d None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:00.250514 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 322/614] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:00.252422 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8c3c3-04b9-458f-abf4-e8157cec415d None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:00.386617 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:00.414166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db298673-aecc-4903-aace-a3321d7b47ae None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:00.471254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8c3c3-04b9-458f-abf4-e8157cec415d None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:00.479303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489974a-fed8-42ed-9834-d968aec11f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:00.480780 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489974a-fed8-42ed-9834-d968aec11f8b None admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:00.481225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489974a-fed8-42ed-9834-d968aec11f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:00.481666 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489974a-fed8-42ed-9834-d968aec11f8b None admin] PATH_INFO: `/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/6e372cc62af24664b01a967435eb8902/roles/21119ebd056740048b9ec639ebb20446` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:00.512459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489974a-fed8-42ed-9834-d968aec11f8b None admin] RBAC: Authorizing `identity:create_grant(project_id=2739f00eb01e46a9acf90ffa360d7d8c, user_id=6e372cc62af24664b01a967435eb8902, role_id=21119ebd056740048b9ec639ebb20446)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:00.513989 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489974a-fed8-42ed-9834-d968aec11f8b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:00.532124 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525ce78-e4d9-422c-b03c-17a235f8def5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:00.532954 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525ce78-e4d9-422c-b03c-17a235f8def5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:00.533147 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525ce78-e4d9-422c-b03c-17a235f8def5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:00.533522 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525ce78-e4d9-422c-b03c-17a235f8def5 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:00.534413 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e525ce78-e4d9-422c-b03c-17a235f8def5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:00.534716 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e525ce78-e4d9-422c-b03c-17a235f8def5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:00.536207 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e525ce78-e4d9-422c-b03c-17a235f8def5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:00.557290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 323/615] 10.222.0.27 () {56 vars in 1135 bytes} [Sat Jul 16 14:50:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:00.558677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0843af8-1847-4fae-b26a-756582863c55 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:00.558991 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0843af8-1847-4fae-b26a-756582863c55 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:00.559284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0843af8-1847-4fae-b26a-756582863c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:00.577111 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f0843af8-1847-4fae-b26a-756582863c55 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:00.578321 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f0843af8-1847-4fae-b26a-756582863c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:00.581619 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:00.599237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 293/616] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:50:00 2022] PUT /identity/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/6e372cc62af24664b01a967435eb8902/roles/21119ebd056740048b9ec639ebb20446 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:50:00.600900 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943519c-83fb-400c-a829-c3165a7bb6dd None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:00.602157 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943519c-83fb-400c-a829-c3165a7bb6dd None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:00.602431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943519c-83fb-400c-a829-c3165a7bb6dd None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:00.604770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 294/617] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:00.610571 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943519c-83fb-400c-a829-c3165a7bb6dd None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:00.695481 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 324/618] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:50:00.698222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386bb50-c0b5-4ed7-b81f-9734ff0883db None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:00.698843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386bb50-c0b5-4ed7-b81f-9734ff0883db None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:00.699464 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386bb50-c0b5-4ed7-b81f-9734ff0883db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:00.731963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-2386bb50-c0b5-4ed7-b81f-9734ff0883db None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:00.735879 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2386bb50-c0b5-4ed7-b81f-9734ff0883db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:00.784428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943519c-83fb-400c-a829-c3165a7bb6dd None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:00.829839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d8ba3-0f87-4788-8233-f73393526da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:00.831214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d8ba3-0f87-4788-8233-f73393526da7 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:00.831737 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d8ba3-0f87-4788-8233-f73393526da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:00.832233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d8ba3-0f87-4788-8233-f73393526da7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:00.837891 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:00.848627 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d8ba3-0f87-4788-8233-f73393526da7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:00.850441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d8ba3-0f87-4788-8233-f73393526da7 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:00.873052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-768d8ba3-0f87-4788-8233-f73393526da7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:00.873052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:00.873052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:00.873052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:00.873052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:00.873052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:00.873052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:00.873052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:00.873052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:00.873052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:00.873052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:00.873052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:00.873052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:00.873052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:00.873052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:00.874642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:00.874642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:00.874642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:00.874642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:00.874642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:00.874642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:00.874642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:00.874642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:00.874642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:00.874642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:00.874642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:00.874642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:00.874642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:00.874642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:00.874642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:00.874642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:00.874642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:00.876938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:00.876938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:00.876938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:00.876938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:00.876938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 295/619] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:00.876938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d8ba3-0f87-4788-8233-f73393526da7 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:00.891434 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 325/620] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:50:00.893055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386bb50-c0b5-4ed7-b81f-9734ff0883db None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:01.043400 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:01.069659 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d8ba3-0f87-4788-8233-f73393526da7 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:01.105599 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386bb50-c0b5-4ed7-b81f-9734ff0883db None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:01.141948 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2118f-1ca1-4863-a082-ffb971110745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:01.143405 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2118f-1ca1-4863-a082-ffb971110745 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:01.143913 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2118f-1ca1-4863-a082-ffb971110745 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:01.145041 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2118f-1ca1-4863-a082-ffb971110745 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:01.170423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2118f-1ca1-4863-a082-ffb971110745 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:50:01.181412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916958e2-5c9d-41de-8a0b-795fa97c30ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:01.183073 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916958e2-5c9d-41de-8a0b-795fa97c30ef None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:01.183840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916958e2-5c9d-41de-8a0b-795fa97c30ef None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:01.184595 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916958e2-5c9d-41de-8a0b-795fa97c30ef None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:01.186362 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916958e2-5c9d-41de-8a0b-795fa97c30ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:01.186970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-9fd2118f-1ca1-4863-a082-ffb971110745 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:50:01.186970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:01.186970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:01.186970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:01.186970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:01.186970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:01.186970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:01.186970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:01.186970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:01.186970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:01.186970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:01.186970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:01.186970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:50:01.186970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:50:01.186970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:50:01.186970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:50:01.188669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:50:01.188669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:50:01.188669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:50:01.188669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:50:01.188669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:01.188669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:01.188669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:01.188669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:01.188669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:01.188669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:01.188669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:01.188669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:01.188669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:01.188669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:01.188669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:01.188669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:01.190218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:01.190218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:01.190218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:01.190218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:01.190218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:50:01.190218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:50:01.190218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:50:01.190218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:50:01.190218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:50:01.190218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:50:01.190218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:50:01.190218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:01.190218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 296/621] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:50:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:01.191563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2118f-1ca1-4863-a082-ffb971110745 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:01.193086 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916958e2-5c9d-41de-8a0b-795fa97c30ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:01.196954 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916958e2-5c9d-41de-8a0b-795fa97c30ef None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:01.227012 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 326/622] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:01.228964 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916958e2-5c9d-41de-8a0b-795fa97c30ef None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:01.305373 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:01.415338 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2118f-1ca1-4863-a082-ffb971110745 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:01.469670 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916958e2-5c9d-41de-8a0b-795fa97c30ef None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:01.489596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105e541-6c05-4980-8e2d-8d1bd6c8dc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:01.491172 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105e541-6c05-4980-8e2d-8d1bd6c8dc01 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:01.491719 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105e541-6c05-4980-8e2d-8d1bd6c8dc01 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:01.492754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105e541-6c05-4980-8e2d-8d1bd6c8dc01 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:01.522229 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a105e541-6c05-4980-8e2d-8d1bd6c8dc01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:50:01.534921 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:01.546220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-a105e541-6c05-4980-8e2d-8d1bd6c8dc01 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:50:01.546220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:01.546220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:01.546220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:01.546220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:01.546220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:01.546220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:01.546220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:01.546220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:01.546220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:01.546220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:01.546220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:01.546220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:50:01.546220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:50:01.546220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:50:01.546220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:50:01.553197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:50:01.553197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:50:01.553197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:50:01.553197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:50:01.553197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:01.553197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:01.553197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:01.553197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:01.553197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:01.553197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:01.553197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:01.553197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:01.553197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:01.553197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:01.553197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:01.553197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:01.555347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:01.555347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:01.555347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:01.555347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:01.555347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:50:01.555347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:50:01.555347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:50:01.555347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:50:01.555347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:50:01.555347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:50:01.555347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:50:01.555347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:01.556583 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a378b-86bb-4758-8b5f-77592e696cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:01.556745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 297/623] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:50:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:01.556869 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a378b-86bb-4758-8b5f-77592e696cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:01.556869 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a378b-86bb-4758-8b5f-77592e696cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:01.556869 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a378b-86bb-4758-8b5f-77592e696cf8 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:01.557201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105e541-6c05-4980-8e2d-8d1bd6c8dc01 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:01.557322 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a378b-86bb-4758-8b5f-77592e696cf8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:01.558714 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a378b-86bb-4758-8b5f-77592e696cf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:01.561998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a378b-86bb-4758-8b5f-77592e696cf8 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:01.582369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 327/624] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:50:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:01.584303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a378b-86bb-4758-8b5f-77592e696cf8 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:01.776447 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105e541-6c05-4980-8e2d-8d1bd6c8dc01 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:01.792556 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:01.799695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a378b-86bb-4758-8b5f-77592e696cf8 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:01.845925 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c39143-a70e-449d-9ede-06951c98d096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:01.847281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c39143-a70e-449d-9ede-06951c98d096 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:01.847780 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c39143-a70e-449d-9ede-06951c98d096 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:01.848276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c39143-a70e-449d-9ede-06951c98d096 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:01.863004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c39143-a70e-449d-9ede-06951c98d096 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:01.865911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c39143-a70e-449d-9ede-06951c98d096 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:01.871350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2b886-12d6-45fe-9c2a-596250c51bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:01.872968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2b886-12d6-45fe-9c2a-596250c51bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:01.873785 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2b886-12d6-45fe-9c2a-596250c51bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:01.874574 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2b886-12d6-45fe-9c2a-596250c51bb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:01.876212 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2b886-12d6-45fe-9c2a-596250c51bb7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:01.885774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2b886-12d6-45fe-9c2a-596250c51bb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:01.889121 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2b886-12d6-45fe-9c2a-596250c51bb7 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:01.894374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-33c39143-a70e-449d-9ede-06951c98d096 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:01.894374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:01.894374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:01.894374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:01.894374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:01.894374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:01.894374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:01.894374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:01.894374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:01.894374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:01.894374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:01.894374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:01.894374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:50:01.894374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:50:01.894374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:50:01.896427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:50:01.896427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:01.896427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:01.896427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:01.896427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:01.896427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:01.896427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:01.896427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:01.896427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:01.896427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:01.896427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:01.896427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:01.896427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:01.896427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:01.896427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:01.896427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:01.896427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:01.898738 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:50:01.898738 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:50:01.898738 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:50:01.898738 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:50:01.898738 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:50:01.898738 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:50:01.898738 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:01.898738 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:01.904194 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 298/625] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:01.905100 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c39143-a70e-449d-9ede-06951c98d096 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:01.918178 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 328/626] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:50:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:01.920089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2b886-12d6-45fe-9c2a-596250c51bb7 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:02.029849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:02.119898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c39143-a70e-449d-9ede-06951c98d096 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:02.120915 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2b886-12d6-45fe-9c2a-596250c51bb7 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:02.175814 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bdbefe-ec59-44c2-845a-c05db46aa7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:02.176630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bdbefe-ec59-44c2-845a-c05db46aa7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:02.176909 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bdbefe-ec59-44c2-845a-c05db46aa7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:02.177182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bdbefe-ec59-44c2-845a-c05db46aa7c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:02.189887 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20d072-6352-4e65-81a4-5634123d3d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:02.191738 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20d072-6352-4e65-81a4-5634123d3d50 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:02.192020 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bdbefe-ec59-44c2-845a-c05db46aa7c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:02.192760 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20d072-6352-4e65-81a4-5634123d3d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:02.193519 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bdbefe-ec59-44c2-845a-c05db46aa7c1 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:02.193716 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20d072-6352-4e65-81a4-5634123d3d50 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:02.196258 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20d072-6352-4e65-81a4-5634123d3d50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:02.197258 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20d072-6352-4e65-81a4-5634123d3d50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:02.200585 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20d072-6352-4e65-81a4-5634123d3d50 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:02.217413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-c6bdbefe-ec59-44c2-845a-c05db46aa7c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:02.217413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:02.217413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:02.217413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:02.217413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:02.217413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:02.217413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:02.217413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:02.217413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:02.217413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:02.217413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:02.217413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:02.217413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:02.217413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:02.217413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:02.218633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:02.218633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:02.218633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:02.218633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:02.218633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:02.218633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:02.218633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:02.218633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:02.218633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:02.218633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:02.218633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:02.218633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:02.218633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:02.218633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:02.218633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:02.218633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:02.218633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:02.220492 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:02.220492 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:02.220492 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:02.220492 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:02.223042 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 299/627] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:02.224233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bdbefe-ec59-44c2-845a-c05db46aa7c1 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:02.229752 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 329/628] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:50:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:02.230712 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20d072-6352-4e65-81a4-5634123d3d50 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:02.256535 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:02.383422 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bdbefe-ec59-44c2-845a-c05db46aa7c1 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:02.387828 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20d072-6352-4e65-81a4-5634123d3d50 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:02.407072 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:02.434404 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ed2ae-cde9-4e56-a0db-102c83717807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:02.435410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ed2ae-cde9-4e56-a0db-102c83717807 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:02.436309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ed2ae-cde9-4e56-a0db-102c83717807 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:02.436697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ed2ae-cde9-4e56-a0db-102c83717807 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:02.437535 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ed2ae-cde9-4e56-a0db-102c83717807 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:02.438127 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ed2ae-cde9-4e56-a0db-102c83717807 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:02.439764 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ed2ae-cde9-4e56-a0db-102c83717807 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:02.449303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4e3dd-fc3b-4e37-be62-c8f276e20799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:02.450756 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4e3dd-fc3b-4e37-be62-c8f276e20799 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:02.451328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4e3dd-fc3b-4e37-be62-c8f276e20799 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:02.452014 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4e3dd-fc3b-4e37-be62-c8f276e20799 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:02.459072 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 330/629] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:02.460353 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ed2ae-cde9-4e56-a0db-102c83717807 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:02.471628 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4e3dd-fc3b-4e37-be62-c8f276e20799 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:02.473668 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4e3dd-fc3b-4e37-be62-c8f276e20799 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:02.501015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-74a4e3dd-fc3b-4e37-be62-c8f276e20799 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:02.501015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:02.501015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:02.501015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:02.501015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:02.501015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:02.501015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:02.501015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:02.501015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:02.501015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:02.501015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:02.501015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:02.501015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:02.501015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:02.501015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:02.502233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:02.502233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:02.502233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:02.502233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:02.502233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:02.502233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:02.502233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:02.502233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:02.502233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:02.502233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:02.502233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:02.502233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:02.502233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:02.502233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:02.502233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:02.502233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:02.502233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:02.503883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:02.503883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:02.503883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:02.503883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:02.506934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 300/630] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:02.510687 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4e3dd-fc3b-4e37-be62-c8f276e20799 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:02.600345 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ed2ae-cde9-4e56-a0db-102c83717807 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:02.643571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6400cf-bade-4f4a-b13a-098f1106bd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:02.644807 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6400cf-bade-4f4a-b13a-098f1106bd58 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:02.645298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6400cf-bade-4f4a-b13a-098f1106bd58 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:02.645796 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6400cf-bade-4f4a-b13a-098f1106bd58 None admin] PATH_INFO: `/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/e1464c83fb294a3d90e5763ef648e834/roles/21119ebd056740048b9ec639ebb20446` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:02.664496 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6400cf-bade-4f4a-b13a-098f1106bd58 None admin] RBAC: Authorizing `identity:create_grant(project_id=2739f00eb01e46a9acf90ffa360d7d8c, user_id=e1464c83fb294a3d90e5763ef648e834, role_id=21119ebd056740048b9ec639ebb20446)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:02.666378 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6400cf-bade-4f4a-b13a-098f1106bd58 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:02.675524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:02.739606 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4e3dd-fc3b-4e37-be62-c8f276e20799 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:02.749720 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 331/631] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:50:02 2022] PUT /identity/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/e1464c83fb294a3d90e5763ef648e834/roles/21119ebd056740048b9ec639ebb20446 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:50:02.751034 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6400cf-bade-4f4a-b13a-098f1106bd58 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:02.790814 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:02.817682 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a5880-98b0-4ab3-b0d8-747db8039a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:02.819249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a5880-98b0-4ab3-b0d8-747db8039a1d None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:02.820556 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a5880-98b0-4ab3-b0d8-747db8039a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:02.820994 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a5880-98b0-4ab3-b0d8-747db8039a1d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:02.837144 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a5880-98b0-4ab3-b0d8-747db8039a1d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:02.839993 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a5880-98b0-4ab3-b0d8-747db8039a1d None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:02.869744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-426a5880-98b0-4ab3-b0d8-747db8039a1d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 16 14:50:02.869744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:02.869744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:02.869744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:02.869744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:02.869744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:02.869744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:02.869744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:02.869744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:02.869744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:02.869744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:02.869744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:02.869744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:50:02.869744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:50:02.869744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:50:02.869744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:50:02.870787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:02.870787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:02.870787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:50:02.870787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:02.870787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:50:02.870787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:02.870787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:02.870787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:02.870787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:02.870787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:02.870787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:02.870787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:02.870787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:02.870787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:02.870787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:02.870787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:02.870787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:02.870787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:02.871775 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:02.871775 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:02.871775 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:50:02.871775 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:50:02.871775 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:50:02.871775 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:50:02.871775 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:50:02.871775 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:50:02.871775 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 16 14:50:02.871775 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:02.872889 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 301/632] 10.222.0.27 () {56 vars in 1093 bytes} [Sat Jul 16 14:50:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:02.884645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a5880-98b0-4ab3-b0d8-747db8039a1d None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:02.916301 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6400cf-bade-4f4a-b13a-098f1106bd58 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:03.003358 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225f243-9c4a-42b3-9f8d-3adfd5a87b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:03.004170 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225f243-9c4a-42b3-9f8d-3adfd5a87b52 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:03.004544 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225f243-9c4a-42b3-9f8d-3adfd5a87b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:03.004904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225f243-9c4a-42b3-9f8d-3adfd5a87b52 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:03.005663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225f243-9c4a-42b3-9f8d-3adfd5a87b52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:03.006004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225f243-9c4a-42b3-9f8d-3adfd5a87b52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:03.010316 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225f243-9c4a-42b3-9f8d-3adfd5a87b52 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:03.055359 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 332/633] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:03.069539 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225f243-9c4a-42b3-9f8d-3adfd5a87b52 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:03.143943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a5880-98b0-4ab3-b0d8-747db8039a1d None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:03.176857 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:03.185001 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee96d6-0b6b-4799-8657-537fb437313f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:03.185697 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee96d6-0b6b-4799-8657-537fb437313f None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:03.192520 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee96d6-0b6b-4799-8657-537fb437313f None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:03.192520 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee96d6-0b6b-4799-8657-537fb437313f None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:03.192520 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee96d6-0b6b-4799-8657-537fb437313f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:03.192520 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee96d6-0b6b-4799-8657-537fb437313f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:03.192520 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee96d6-0b6b-4799-8657-537fb437313f None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:03.222973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 302/634] 10.222.0.27 () {56 vars in 1133 bytes} [Sat Jul 16 14:50:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:03.222973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ca440-71ac-49f3-b8e4-ecc7c35e3617 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:03.222973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ca440-71ac-49f3-b8e4-ecc7c35e3617 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:03.222973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ca440-71ac-49f3-b8e4-ecc7c35e3617 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:03.222973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 303/635] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:03.224200 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf1cbb-c605-4ee6-992b-3688e5866ce2 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:03.224479 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf1cbb-c605-4ee6-992b-3688e5866ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:03.224806 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf1cbb-c605-4ee6-992b-3688e5866ce2 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:03.225292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:03.229301 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 304/636] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:03.229478 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8364ee-1ab3-4044-b29b-cb56034e7a78 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:03.229711 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8364ee-1ab3-4044-b29b-cb56034e7a78 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:03.230573 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8364ee-1ab3-4044-b29b-cb56034e7a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:03.251190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225f243-9c4a-42b3-9f8d-3adfd5a87b52 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:03.261717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-4c8364ee-1ab3-4044-b29b-cb56034e7a78 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:03.264603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-4c8364ee-1ab3-4044-b29b-cb56034e7a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:03.299849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ed253-be56-46cf-932a-fc48610bb44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:03.300672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ed253-be56-46cf-932a-fc48610bb44f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:03.300963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ed253-be56-46cf-932a-fc48610bb44f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:03.301233 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ed253-be56-46cf-932a-fc48610bb44f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:03.312956 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ed253-be56-46cf-932a-fc48610bb44f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:03.314557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ed253-be56-46cf-932a-fc48610bb44f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:03.331960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-d63ed253-be56-46cf-932a-fc48610bb44f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 16 14:50:03.331960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:03.331960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:03.331960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:03.331960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:03.331960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:03.331960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:03.331960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:03.331960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:03.331960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:03.331960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:03.331960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:03.331960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:50:03.331960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:50:03.331960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:50:03.331960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:50:03.333364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:03.333364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:03.333364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:50:03.333364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:03.333364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:50:03.333364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:03.333364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:03.333364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:03.333364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:03.333364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:03.333364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:03.333364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:03.333364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:03.333364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:03.333364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:03.333364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:03.333364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:03.333364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:03.335030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:03.335030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:03.335030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:50:03.335030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:50:03.335030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:50:03.335030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:50:03.335030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:50:03.335030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:50:03.335030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 16 14:50:03.335030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:03.335030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 333/637] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:50:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:03.335030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ed253-be56-46cf-932a-fc48610bb44f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:03.395840 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 305/638] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 14:50:03.395840 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dea9d3-c4a6-424c-8589-c02174866f7c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:03.395840 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dea9d3-c4a6-424c-8589-c02174866f7c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:03.395840 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dea9d3-c4a6-424c-8589-c02174866f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:03.417038 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-09dea9d3-c4a6-424c-8589-c02174866f7c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:03.419167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-09dea9d3-c4a6-424c-8589-c02174866f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:03.502820 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ed253-be56-46cf-932a-fc48610bb44f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:03.518927 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 306/639] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:50:03.518927 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dea9d3-c4a6-424c-8589-c02174866f7c None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:03.563380 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dba98f-faa5-4ef4-8cb0-0301f35f228d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:03.564462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dba98f-faa5-4ef4-8cb0-0301f35f228d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:03.564961 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dba98f-faa5-4ef4-8cb0-0301f35f228d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:03.565451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dba98f-faa5-4ef4-8cb0-0301f35f228d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:03.587977 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:03.591470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:03.653430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dea9d3-c4a6-424c-8589-c02174866f7c None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:03.678323 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dba98f-faa5-4ef4-8cb0-0301f35f228d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:03.681774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dba98f-faa5-4ef4-8cb0-0301f35f228d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:03.698100 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28113d9c-0885-4c20-8c1e-216860f60d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:03.698787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28113d9c-0885-4c20-8c1e-216860f60d1b None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:03.699171 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28113d9c-0885-4c20-8c1e-216860f60d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:03.699446 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28113d9c-0885-4c20-8c1e-216860f60d1b None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:03.700305 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28113d9c-0885-4c20-8c1e-216860f60d1b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:03.700613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28113d9c-0885-4c20-8c1e-216860f60d1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:03.702072 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28113d9c-0885-4c20-8c1e-216860f60d1b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:03.730217 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 307/640] 10.222.0.27 () {56 vars in 1137 bytes} [Sat Jul 16 14:50:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:03.730217 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28113d9c-0885-4c20-8c1e-216860f60d1b None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:03.846842 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28113d9c-0885-4c20-8c1e-216860f60d1b None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:03.856151 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:03.874974 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 334/641] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:50:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:50:03.874974 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dba98f-faa5-4ef4-8cb0-0301f35f228d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:03.910866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301db593-366e-4d26-8c2b-19abcab3661e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:03.911989 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301db593-366e-4d26-8c2b-19abcab3661e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:03.912274 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301db593-366e-4d26-8c2b-19abcab3661e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:03.912554 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301db593-366e-4d26-8c2b-19abcab3661e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:03.929233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301db593-366e-4d26-8c2b-19abcab3661e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:50:03.937505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-301db593-366e-4d26-8c2b-19abcab3661e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:50:03.937505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:03.937505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:03.937505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:03.937505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:03.937505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:03.937505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:03.937505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:03.937505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:03.937505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:03.937505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:03.937505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:03.937505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:50:03.937505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:50:03.937505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:50:03.937505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:50:03.939276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:50:03.939276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:50:03.939276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:50:03.939276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:50:03.939276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:03.939276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:03.939276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:03.939276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:03.939276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:03.939276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:03.939276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:03.939276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:03.939276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:03.939276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:03.939276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:03.939276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:03.940546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:03.940546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:03.940546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:03.940546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:03.940546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:50:03.940546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:50:03.940546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:50:03.940546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:50:03.940546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:50:03.940546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:50:03.940546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:50:03.940546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:03.943565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 308/642] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:50:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:03.944909 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301db593-366e-4d26-8c2b-19abcab3661e None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:03.950450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:04.054694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dba98f-faa5-4ef4-8cb0-0301f35f228d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:04.090942 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301db593-366e-4d26-8c2b-19abcab3661e None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:04.113821 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a36172-df3e-4acd-9727-38364aae2f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:04.115049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a36172-df3e-4acd-9727-38364aae2f80 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:04.115472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a36172-df3e-4acd-9727-38364aae2f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:04.115922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a36172-df3e-4acd-9727-38364aae2f80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:04.132048 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a36172-df3e-4acd-9727-38364aae2f80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:50:04.142074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-56a36172-df3e-4acd-9727-38364aae2f80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:50:04.142074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:04.142074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:04.142074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:04.142074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:04.142074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:04.142074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:04.142074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:04.142074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:04.142074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:04.142074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:04.142074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:04.142074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:50:04.142074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:50:04.142074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:50:04.142074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:50:04.144079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:50:04.144079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:50:04.144079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:50:04.144079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:50:04.144079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:04.144079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:04.144079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:04.144079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:04.144079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:04.144079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:04.144079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:04.144079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:04.144079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:04.144079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:04.144079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:04.144079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:04.145717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:04.145717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:04.145717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:04.145717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:04.145717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:50:04.145717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:50:04.145717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:50:04.145717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:50:04.145717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:50:04.145717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:50:04.145717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:50:04.145717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:04.145717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 335/643] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:50:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:04.145717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a36172-df3e-4acd-9727-38364aae2f80 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:04.167461 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87836d9-cb6c-4e19-b54e-8fddcd48798c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:04.169054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87836d9-cb6c-4e19-b54e-8fddcd48798c None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:04.169820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87836d9-cb6c-4e19-b54e-8fddcd48798c None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:04.170523 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87836d9-cb6c-4e19-b54e-8fddcd48798c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:04.231876 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:04.297614 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87836d9-cb6c-4e19-b54e-8fddcd48798c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:04.300212 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87836d9-cb6c-4e19-b54e-8fddcd48798c None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:04.323681 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a36172-df3e-4acd-9727-38364aae2f80 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:04.327448 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:04.373191 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0636045-2230-49da-97d1-53525b319f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:04.374646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0636045-2230-49da-97d1-53525b319f1d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:04.375187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0636045-2230-49da-97d1-53525b319f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:04.375701 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0636045-2230-49da-97d1-53525b319f1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:04.401546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0636045-2230-49da-97d1-53525b319f1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:04.403833 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0636045-2230-49da-97d1-53525b319f1d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:04.434007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-d0636045-2230-49da-97d1-53525b319f1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:04.434007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:04.434007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:04.434007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:04.434007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:04.434007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:04.434007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:04.434007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:04.434007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:04.434007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:04.434007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:04.434007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:04.434007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:04.434007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:04.434007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:04.435257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:04.435257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:04.435257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:04.435257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:04.435257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:04.435257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:04.435257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:04.435257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:04.435257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:04.435257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:04.435257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:04.435257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:04.435257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:04.435257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:04.435257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:04.435257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:04.435257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:04.441411 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:04.441411 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:04.441411 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:04.441411 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:04.441411 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 336/644] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:04.441411 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0636045-2230-49da-97d1-53525b319f1d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:04.478168 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 309/645] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:50:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:50:04.478168 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87836d9-cb6c-4e19-b54e-8fddcd48798c None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:04.580937 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:04.632172 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87836d9-cb6c-4e19-b54e-8fddcd48798c None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:04.636375 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0636045-2230-49da-97d1-53525b319f1d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:04.691188 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd223227-7290-4ef0-a4e6-09ede74dd385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:04.692069 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd223227-7290-4ef0-a4e6-09ede74dd385 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:04.692520 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd223227-7290-4ef0-a4e6-09ede74dd385 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:04.692978 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd223227-7290-4ef0-a4e6-09ede74dd385 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:04.694215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd223227-7290-4ef0-a4e6-09ede74dd385 None admin] RBAC: Adding query filter params (name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:04.694934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd223227-7290-4ef0-a4e6-09ede74dd385 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:04.697839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd223227-7290-4ef0-a4e6-09ede74dd385 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:04.700855 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956ed99-adff-43ae-a778-3d9e474370ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:04.701912 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956ed99-adff-43ae-a778-3d9e474370ac None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:04.702374 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956ed99-adff-43ae-a778-3d9e474370ac None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:04.702822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956ed99-adff-43ae-a778-3d9e474370ac None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:04.704137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956ed99-adff-43ae-a778-3d9e474370ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:04.704634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956ed99-adff-43ae-a778-3d9e474370ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:04.707129 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4956ed99-adff-43ae-a778-3d9e474370ac None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:04.714444 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 310/646] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:50:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:04.715918 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd223227-7290-4ef0-a4e6-09ede74dd385 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:04.736024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 337/647] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:50:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:04.736024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956ed99-adff-43ae-a778-3d9e474370ac None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:04.768820 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:04.913218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd223227-7290-4ef0-a4e6-09ede74dd385 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:04.915823 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956ed99-adff-43ae-a778-3d9e474370ac None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:04.962555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:04.975392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f22e0c-510a-42a7-bf5e-5aea29c3c1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:04.975923 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a483d0c-5236-4619-a6a2-402ff6226eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:04.976464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f22e0c-510a-42a7-bf5e-5aea29c3c1af None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:04.976898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f22e0c-510a-42a7-bf5e-5aea29c3c1af None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:04.977240 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a483d0c-5236-4619-a6a2-402ff6226eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:04.977359 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f22e0c-510a-42a7-bf5e-5aea29c3c1af None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:04.977890 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a483d0c-5236-4619-a6a2-402ff6226eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:04.978330 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a483d0c-5236-4619-a6a2-402ff6226eb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:04.978566 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f22e0c-510a-42a7-bf5e-5aea29c3c1af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:04.979086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f22e0c-510a-42a7-bf5e-5aea29c3c1af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:04.984825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f22e0c-510a-42a7-bf5e-5aea29c3c1af None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:04.996889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a483d0c-5236-4619-a6a2-402ff6226eb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:04.999097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a483d0c-5236-4619-a6a2-402ff6226eb9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:05.011470 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 311/648] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:05.012901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f22e0c-510a-42a7-bf5e-5aea29c3c1af None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:05.024751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-3a483d0c-5236-4619-a6a2-402ff6226eb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:05.024751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:05.024751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:05.024751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:05.024751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:05.024751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:05.024751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:05.024751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:05.024751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:05.024751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:05.024751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:05.024751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:05.024751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:05.024751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:05.024751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:05.025719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:05.025719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:05.025719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:05.025719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:05.025719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:05.025719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:05.025719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:05.025719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:05.025719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:05.025719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:05.025719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:05.025719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:05.025719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:05.025719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:05.025719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:05.025719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:05.025719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:05.026620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:05.026620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:05.026620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:05.026620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:05.026620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 338/649] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:05.027376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a483d0c-5236-4619-a6a2-402ff6226eb9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:05.141067 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:05.141906 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f22e0c-510a-42a7-bf5e-5aea29c3c1af None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:05.177451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a483d0c-5236-4619-a6a2-402ff6226eb9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:05.184368 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c503b-76cc-4e66-88cf-66b24f1911cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:05.185175 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c503b-76cc-4e66-88cf-66b24f1911cf None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:05.185454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c503b-76cc-4e66-88cf-66b24f1911cf None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:05.185725 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c503b-76cc-4e66-88cf-66b24f1911cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:05.196816 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c503b-76cc-4e66-88cf-66b24f1911cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:05.198283 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c503b-76cc-4e66-88cf-66b24f1911cf None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:05.214941 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0298766-9b09-45f2-906c-a0f45fb4f546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:05.215748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0298766-9b09-45f2-906c-a0f45fb4f546 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:05.216030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0298766-9b09-45f2-906c-a0f45fb4f546 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:05.216306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0298766-9b09-45f2-906c-a0f45fb4f546 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:05.228616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0298766-9b09-45f2-906c-a0f45fb4f546 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:05.230082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0298766-9b09-45f2-906c-a0f45fb4f546 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:05.230286 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-b97c503b-76cc-4e66-88cf-66b24f1911cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:05.230286 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:05.230286 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:05.230286 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:05.230286 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:05.230286 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:05.230286 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:05.230286 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:05.230286 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:05.230286 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:05.230286 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:05.230286 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:05.230286 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:05.230286 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:05.230286 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:05.231342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:05.231342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:05.231342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:05.231342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:05.231342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:05.231342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:05.231342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:05.231342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:05.231342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:05.231342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:05.231342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:05.231342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:05.231342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:05.231342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:05.231342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:05.231342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:05.231342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:05.232428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:05.232428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:05.232428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:05.232428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:05.232702 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 312/650] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:05.234112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c503b-76cc-4e66-88cf-66b24f1911cf None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:05.265227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-b0298766-9b09-45f2-906c-a0f45fb4f546 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:05.265227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:05.265227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:05.265227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:05.265227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:05.265227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:05.265227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:05.265227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:05.265227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:05.265227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:05.265227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:05.265227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:05.265227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:05.265227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:05.265227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:05.267231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:05.267231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:05.267231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:05.267231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:05.267231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:05.267231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:05.267231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:05.267231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:05.267231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:05.267231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:05.267231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:05.267231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:05.267231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:05.267231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:05.267231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:05.267231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:05.267231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:05.269299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:05.269299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:05.269299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:05.269299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:05.269299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 339/651] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:05.269299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0298766-9b09-45f2-906c-a0f45fb4f546 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:05.296382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:05.380212 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c503b-76cc-4e66-88cf-66b24f1911cf None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:05.396902 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0298766-9b09-45f2-906c-a0f45fb4f546 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:05.418485 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d04f5-d952-44c3-81ca-35f2dde7b7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:05.419335 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d04f5-d952-44c3-81ca-35f2dde7b7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:05.419611 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d04f5-d952-44c3-81ca-35f2dde7b7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:05.419880 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d04f5-d952-44c3-81ca-35f2dde7b7f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:05.428354 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d04f5-d952-44c3-81ca-35f2dde7b7f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:05.431645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d04f5-d952-44c3-81ca-35f2dde7b7f0 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:05.433732 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:05.448204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-9a9d04f5-d952-44c3-81ca-35f2dde7b7f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:05.448204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:05.448204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:05.448204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:05.448204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:05.448204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:05.448204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:05.448204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:05.448204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:05.448204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:05.448204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:05.448204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:05.448204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:50:05.448204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:50:05.448204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:50:05.449225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:50:05.449225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:05.449225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:05.449225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:05.449225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:05.449225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:05.449225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:05.449225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:05.449225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:05.449225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:05.449225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:05.449225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:05.449225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:05.449225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:05.449225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:05.449225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:05.449225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:05.450208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:50:05.450208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:50:05.450208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:50:05.450208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:50:05.450208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:50:05.450208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:50:05.450208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:05.450208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:05.450816 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f38bf-6021-4765-ae4b-09ef693b66c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:05.450816 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f38bf-6021-4765-ae4b-09ef693b66c4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:05.451138 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 313/652] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:05.451642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f38bf-6021-4765-ae4b-09ef693b66c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:05.452177 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f38bf-6021-4765-ae4b-09ef693b66c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:05.452554 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987ce0d-3285-4718-95a5-ef8d3de183b1 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:05.452842 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987ce0d-3285-4718-95a5-ef8d3de183b1 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:05.453195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f987ce0d-3285-4718-95a5-ef8d3de183b1 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:05.453802 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f38bf-6021-4765-ae4b-09ef693b66c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:05.454350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f38bf-6021-4765-ae4b-09ef693b66c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:05.456255 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 314/653] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:05.456255 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987ce0d-3285-4718-95a5-ef8d3de183b1 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:05.457096 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f38bf-6021-4765-ae4b-09ef693b66c4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:05.490670 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 340/654] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:05.490670 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f38bf-6021-4765-ae4b-09ef693b66c4 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:05.585180 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:05.590380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f987ce0d-3285-4718-95a5-ef8d3de183b1 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:05.628581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae8b7b-8c41-4a8e-ad49-05cd5a549d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:05.629257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae8b7b-8c41-4a8e-ad49-05cd5a549d63 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:05.629535 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae8b7b-8c41-4a8e-ad49-05cd5a549d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:05.629809 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae8b7b-8c41-4a8e-ad49-05cd5a549d63 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:05.630541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae8b7b-8c41-4a8e-ad49-05cd5a549d63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:05.630850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae8b7b-8c41-4a8e-ad49-05cd5a549d63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:05.632303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae8b7b-8c41-4a8e-ad49-05cd5a549d63 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:05.645063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 315/655] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:05.646350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae8b7b-8c41-4a8e-ad49-05cd5a549d63 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:05.649499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f38bf-6021-4765-ae4b-09ef693b66c4 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:05.710247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323c8a5-3f5c-49f7-81ac-789f30f164ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:05.712238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323c8a5-3f5c-49f7-81ac-789f30f164ac None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:05.712666 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323c8a5-3f5c-49f7-81ac-789f30f164ac None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:05.713132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323c8a5-3f5c-49f7-81ac-789f30f164ac None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:05.714017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323c8a5-3f5c-49f7-81ac-789f30f164ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:05.714322 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323c8a5-3f5c-49f7-81ac-789f30f164ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:05.716785 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323c8a5-3f5c-49f7-81ac-789f30f164ac None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:05.737605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 341/656] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:05.739917 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645751e0-8093-49c3-aff6-f980e177887d None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:05.740409 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645751e0-8093-49c3-aff6-f980e177887d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:05.740906 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645751e0-8093-49c3-aff6-f980e177887d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:05.760850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-645751e0-8093-49c3-aff6-f980e177887d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:05.762190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-645751e0-8093-49c3-aff6-f980e177887d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:05.770559 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae8b7b-8c41-4a8e-ad49-05cd5a549d63 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:05.807556 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:05.812542 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739401d3-9327-46ed-aecd-0da89bff4183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:05.813353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739401d3-9327-46ed-aecd-0da89bff4183 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:05.813699 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739401d3-9327-46ed-aecd-0da89bff4183 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:05.814125 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739401d3-9327-46ed-aecd-0da89bff4183 None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:05.815103 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739401d3-9327-46ed-aecd-0da89bff4183 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:05.815455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739401d3-9327-46ed-aecd-0da89bff4183 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:05.817346 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739401d3-9327-46ed-aecd-0da89bff4183 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:05.832315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 316/657] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:50:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:05.833184 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739401d3-9327-46ed-aecd-0da89bff4183 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:05.846486 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 342/658] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:50:05.847379 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645751e0-8093-49c3-aff6-f980e177887d None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:05.875238 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:05.978058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739401d3-9327-46ed-aecd-0da89bff4183 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:06.021350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645751e0-8093-49c3-aff6-f980e177887d None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:06.031313 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d85ea-a356-498a-8721-8e2ef60661cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:06.032605 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d85ea-a356-498a-8721-8e2ef60661cd None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:06.033058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d85ea-a356-498a-8721-8e2ef60661cd None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:06.033526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d85ea-a356-498a-8721-8e2ef60661cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:06.044823 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d85ea-a356-498a-8721-8e2ef60661cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:06.046779 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d85ea-a356-498a-8721-8e2ef60661cd None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:06.064329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-8d7d85ea-a356-498a-8721-8e2ef60661cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:06.064329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:06.064329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:06.064329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:06.064329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:06.064329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:06.064329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:06.064329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:06.064329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:06.064329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:06.064329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:06.064329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:06.064329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:50:06.064329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:50:06.064329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:50:06.066112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:50:06.066112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:06.066112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:06.066112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:06.066112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:06.066112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:06.066112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:06.066112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:06.066112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:06.066112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:06.066112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:06.066112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:06.066112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:06.066112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:06.066112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:06.066112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:06.066112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:06.067926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:50:06.067926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:50:06.067926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:50:06.067926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:50:06.067926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:50:06.067926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:50:06.067926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:06.067926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:06.067926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 317/659] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:06.067926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d85ea-a356-498a-8721-8e2ef60661cd None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:06.075168 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674afdc0-bf26-4ec3-9ce5-fc5eb0023e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:06.076106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674afdc0-bf26-4ec3-9ce5-fc5eb0023e0f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:06.076475 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674afdc0-bf26-4ec3-9ce5-fc5eb0023e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:06.076807 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674afdc0-bf26-4ec3-9ce5-fc5eb0023e0f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:06.085162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674afdc0-bf26-4ec3-9ce5-fc5eb0023e0f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:06.087470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674afdc0-bf26-4ec3-9ce5-fc5eb0023e0f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:06.103707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-674afdc0-bf26-4ec3-9ce5-fc5eb0023e0f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 16 14:50:06.103707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:06.103707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:06.103707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:06.103707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:06.103707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:06.103707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:06.103707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:06.103707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:06.103707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:06.103707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:06.103707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:06.103707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:50:06.103707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:50:06.103707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:50:06.103707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:50:06.105652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:06.105652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:06.105652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:50:06.105652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:06.105652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:50:06.105652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:06.105652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:06.105652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:06.105652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:06.105652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:06.105652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:06.105652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:06.105652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:06.105652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:06.105652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:06.105652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:06.105652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:06.105652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:06.107530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:06.107530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:06.107530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:50:06.107530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:50:06.107530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:50:06.107530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:50:06.107530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:50:06.107530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:50:06.107530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 16 14:50:06.107530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:06.107530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 343/660] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:50:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:06.107530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674afdc0-bf26-4ec3-9ce5-fc5eb0023e0f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:06.152201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:06.176338 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:06.198878 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d85ea-a356-498a-8721-8e2ef60661cd None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:06.244747 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1299167d-5605-46a6-acfd-b592c77875b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:06.245999 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299167d-5605-46a6-acfd-b592c77875b1 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:06.246488 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299167d-5605-46a6-acfd-b592c77875b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:06.246995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299167d-5605-46a6-acfd-b592c77875b1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:06.253064 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674afdc0-bf26-4ec3-9ce5-fc5eb0023e0f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:06.261759 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1299167d-5605-46a6-acfd-b592c77875b1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:06.264518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1299167d-5605-46a6-acfd-b592c77875b1 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:06.287567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-1299167d-5605-46a6-acfd-b592c77875b1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 16 14:50:06.287567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:06.287567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:06.287567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:06.287567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:06.287567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:06.287567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:06.287567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:06.287567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:06.287567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:06.287567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:06.287567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:06.287567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:50:06.287567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:50:06.287567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:50:06.287567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:50:06.288731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:06.288731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:06.288731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:50:06.288731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:06.288731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:50:06.288731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:06.288731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:06.288731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:06.288731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:06.288731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:06.288731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:06.288731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:06.288731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:06.288731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:06.288731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:06.288731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:06.288731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:06.288731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:06.289835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:06.289835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:06.289835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:50:06.289835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:50:06.289835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:50:06.289835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:50:06.289835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:50:06.289835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:50:06.289835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 16 14:50:06.289835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:06.289835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 318/661] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:50:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:06.290954 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1299167d-5605-46a6-acfd-b592c77875b1 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:06.303725 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbd3ac-d8c8-40fc-a2ec-388ed51c59ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:06.306344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbd3ac-d8c8-40fc-a2ec-388ed51c59ae None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:06.306847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbd3ac-d8c8-40fc-a2ec-388ed51c59ae None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:06.307378 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbd3ac-d8c8-40fc-a2ec-388ed51c59ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:06.308971 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbd3ac-d8c8-40fc-a2ec-388ed51c59ae None admin] RBAC: Adding query filter params (role.id=21119ebd056740048b9ec639ebb20446, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=3f2e241494244edf8d283e8f44192c46) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:06.309513 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbd3ac-d8c8-40fc-a2ec-388ed51c59ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:06.312332 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bbd3ac-d8c8-40fc-a2ec-388ed51c59ae None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:06.337984 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 344/662] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:50:06 2022] GET /identity/v3/role_assignments?user.id=3f2e241494244edf8d283e8f44192c46&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=21119ebd056740048b9ec639ebb20446 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:06.337984 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbd3ac-d8c8-40fc-a2ec-388ed51c59ae None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:06.439696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1299167d-5605-46a6-acfd-b592c77875b1 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:06.483909 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbaea5-9f10-4151-b35a-c0f81d6ea09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:06.484735 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbaea5-9f10-4151-b35a-c0f81d6ea09d None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:06.485010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbaea5-9f10-4151-b35a-c0f81d6ea09d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:06.485281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbaea5-9f10-4151-b35a-c0f81d6ea09d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:06.495812 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:06.499787 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:06.500566 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbaea5-9f10-4151-b35a-c0f81d6ea09d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:50:06.506468 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbd3ac-d8c8-40fc-a2ec-388ed51c59ae None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:06.518603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-cbbbaea5-9f10-4151-b35a-c0f81d6ea09d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:50:06.518603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:06.518603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:06.518603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:06.518603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:06.518603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:06.518603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:06.518603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:06.518603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:06.518603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:06.518603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:06.518603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:06.518603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:50:06.518603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:50:06.518603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:50:06.518603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:50:06.520289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:50:06.520289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:50:06.520289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:50:06.520289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:50:06.520289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:06.520289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:06.520289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:06.520289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:06.520289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:06.520289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:06.520289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:06.520289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:06.520289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:06.520289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:06.520289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:06.520289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:06.521772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:06.521772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:06.521772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:06.521772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:06.521772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:50:06.521772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:50:06.521772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:50:06.521772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:50:06.521772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:50:06.521772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:50:06.521772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:50:06.521772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:06.521772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 319/663] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:50:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:06.523581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbaea5-9f10-4151-b35a-c0f81d6ea09d None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:06.566239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c41389-7d65-4466-8343-8c149fe8ede9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:06.566932 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c41389-7d65-4466-8343-8c149fe8ede9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:06.567212 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c41389-7d65-4466-8343-8c149fe8ede9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:06.567483 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c41389-7d65-4466-8343-8c149fe8ede9 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:06.568277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c41389-7d65-4466-8343-8c149fe8ede9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:06.568582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c41389-7d65-4466-8343-8c149fe8ede9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:06.570571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c41389-7d65-4466-8343-8c149fe8ede9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:06.588667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 345/664] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:50:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:06.589623 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c41389-7d65-4466-8343-8c149fe8ede9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:06.694612 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbaea5-9f10-4151-b35a-c0f81d6ea09d None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:06.740742 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c41389-7d65-4466-8343-8c149fe8ede9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:06.751086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1459b-c136-42df-b965-e922cda176cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:06.752164 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1459b-c136-42df-b965-e922cda176cb None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:06.752518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1459b-c136-42df-b965-e922cda176cb None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:06.753013 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1459b-c136-42df-b965-e922cda176cb None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:06.754275 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1459b-c136-42df-b965-e922cda176cb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:06.754812 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1459b-c136-42df-b965-e922cda176cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:06.758218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1459b-c136-42df-b965-e922cda176cb None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:06.786936 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 320/665] 10.222.0.27 () {56 vars in 1139 bytes} [Sat Jul 16 14:50:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:06.787413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1459b-c136-42df-b965-e922cda176cb None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:06.794241 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe78f1-1ef5-4afe-9f09-4f35e5e12b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:06.794965 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe78f1-1ef5-4afe-9f09-4f35e5e12b30 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:06.795253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe78f1-1ef5-4afe-9f09-4f35e5e12b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:06.795534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe78f1-1ef5-4afe-9f09-4f35e5e12b30 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:06.796398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe78f1-1ef5-4afe-9f09-4f35e5e12b30 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:06.796853 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe78f1-1ef5-4afe-9f09-4f35e5e12b30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:06.799640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe78f1-1ef5-4afe-9f09-4f35e5e12b30 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:06.827347 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 346/666] 10.222.0.27 () {56 vars in 1137 bytes} [Sat Jul 16 14:50:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:06.828672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe78f1-1ef5-4afe-9f09-4f35e5e12b30 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:06.856520 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:06.867551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:06.935232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1459b-c136-42df-b965-e922cda176cb None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:06.972295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe78f1-1ef5-4afe-9f09-4f35e5e12b30 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:06.984715 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc1060-750f-4859-b3af-05e6c27532d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:06.985838 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc1060-750f-4859-b3af-05e6c27532d8 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:06.986328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc1060-750f-4859-b3af-05e6c27532d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:06.986806 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc1060-750f-4859-b3af-05e6c27532d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:06.988185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc1060-750f-4859-b3af-05e6c27532d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:06.988719 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc1060-750f-4859-b3af-05e6c27532d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:06.991251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc1060-750f-4859-b3af-05e6c27532d8 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:07.005672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 321/667] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:50:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:07.006729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc1060-750f-4859-b3af-05e6c27532d8 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:07.023689 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45eace-d26f-4613-b26b-aac42b73fcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:07.025156 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45eace-d26f-4613-b26b-aac42b73fcda None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:07.025506 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45eace-d26f-4613-b26b-aac42b73fcda None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:07.025854 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45eace-d26f-4613-b26b-aac42b73fcda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:07.026759 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b45eace-d26f-4613-b26b-aac42b73fcda None admin] RBAC: Adding query filter params (role.id=21119ebd056740048b9ec639ebb20446, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=7d689162291f4f4b97e886ea20d3a8b2) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:07.027088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b45eace-d26f-4613-b26b-aac42b73fcda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:07.028902 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b45eace-d26f-4613-b26b-aac42b73fcda None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:07.035956 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 347/668] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:50:06 2022] GET /identity/v3/role_assignments?user.id=7d689162291f4f4b97e886ea20d3a8b2&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=21119ebd056740048b9ec639ebb20446 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:07.036850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45eace-d26f-4613-b26b-aac42b73fcda None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:07.148342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc1060-750f-4859-b3af-05e6c27532d8 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:07.174105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:07.189743 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:07.192468 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d74b0-ce33-44a8-99be-2ac2103aa8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:07.193161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d74b0-ce33-44a8-99be-2ac2103aa8ce None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:07.193610 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d74b0-ce33-44a8-99be-2ac2103aa8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:07.193794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d74b0-ce33-44a8-99be-2ac2103aa8ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:07.224386 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45eace-d26f-4613-b26b-aac42b73fcda None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:07.291625 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55239dd4-e262-4061-9859-f107983e49c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:07.293525 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55239dd4-e262-4061-9859-f107983e49c1 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:07.294268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55239dd4-e262-4061-9859-f107983e49c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:07.294750 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55239dd4-e262-4061-9859-f107983e49c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:07.302508 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d74b0-ce33-44a8-99be-2ac2103aa8ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:07.302508 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d74b0-ce33-44a8-99be-2ac2103aa8ce None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:07.405827 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55239dd4-e262-4061-9859-f107983e49c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:07.408242 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55239dd4-e262-4061-9859-f107983e49c1 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:07.485256 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 322/669] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:50:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:50:07.486850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d74b0-ce33-44a8-99be-2ac2103aa8ce None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:07.532947 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:07.540344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:07.579386 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 348/670] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:50:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:50:07.581161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55239dd4-e262-4061-9859-f107983e49c1 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:07.711427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d74b0-ce33-44a8-99be-2ac2103aa8ce None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:07.768069 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55239dd4-e262-4061-9859-f107983e49c1 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:07.800234 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7eb9d-0cbf-4886-a65d-4f2a02c6aab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:07.802769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7eb9d-0cbf-4886-a65d-4f2a02c6aab2 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:07.803521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7eb9d-0cbf-4886-a65d-4f2a02c6aab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:07.804595 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7eb9d-0cbf-4886-a65d-4f2a02c6aab2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:07.821499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3ebab-511b-4068-b618-44695ac4f540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:07.822685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3ebab-511b-4068-b618-44695ac4f540 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:07.823126 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3ebab-511b-4068-b618-44695ac4f540 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:07.823399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3ebab-511b-4068-b618-44695ac4f540 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:07.824511 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d7eb9d-0cbf-4886-a65d-4f2a02c6aab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:07.826592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d7eb9d-0cbf-4886-a65d-4f2a02c6aab2 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:07.843114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3ebab-511b-4068-b618-44695ac4f540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:07.845553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3ebab-511b-4068-b618-44695ac4f540 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:07.848384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-26d7eb9d-0cbf-4886-a65d-4f2a02c6aab2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:07.848384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:07.848384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:07.848384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:07.848384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:07.848384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:07.848384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:07.848384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:07.848384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:07.848384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:07.848384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:07.848384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:07.848384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:07.848384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:07.848384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:07.849386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:07.849386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:07.849386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:07.849386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:07.849386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:07.849386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:07.849386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:07.849386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:07.849386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:07.849386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:07.849386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:07.849386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:07.849386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:07.849386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:07.849386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:07.849386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:07.849386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:07.850400 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:07.850400 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:07.850400 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:07.850400 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:07.858414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 323/671] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:07.859569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7eb9d-0cbf-4886-a65d-4f2a02c6aab2 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:07.895268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-d7c3ebab-511b-4068-b618-44695ac4f540 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:07.895268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:07.895268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:07.895268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:07.895268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:07.895268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:07.895268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:07.895268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:07.895268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:07.895268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:07.895268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:07.895268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:07.895268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:07.895268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:07.895268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:07.896357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:07.896357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:07.896357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:07.896357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:07.896357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:07.896357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:07.896357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:07.896357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:07.896357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:07.896357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:07.896357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:07.896357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:07.896357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:07.896357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:07.896357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:07.896357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:07.896357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:07.897394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:07.897394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:07.897394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:07.897394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:07.908223 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 349/672] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:07.909971 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3ebab-511b-4068-b618-44695ac4f540 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:07.949722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:07.968185 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:07.993271 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7eb9d-0cbf-4886-a65d-4f2a02c6aab2 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:08.055070 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f88bc3-e84d-4812-802e-f1a6485cd067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:08.058185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f88bc3-e84d-4812-802e-f1a6485cd067 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:08.059364 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f88bc3-e84d-4812-802e-f1a6485cd067 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:08.060212 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f88bc3-e84d-4812-802e-f1a6485cd067 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:08.083611 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f88bc3-e84d-4812-802e-f1a6485cd067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:08.086646 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f88bc3-e84d-4812-802e-f1a6485cd067 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:08.100635 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3ebab-511b-4068-b618-44695ac4f540 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:08.125179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-24f88bc3-e84d-4812-802e-f1a6485cd067 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:08.125179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:08.125179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:08.125179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:08.125179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:08.125179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:08.125179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:08.125179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:08.125179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:08.125179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:08.125179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:08.125179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:08.125179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:08.125179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:08.125179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:08.126976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:08.126976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:08.126976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:08.126976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:08.126976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:08.126976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:08.126976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:08.126976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:08.126976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:08.126976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:08.126976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:08.126976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:08.126976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:08.126976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:08.126976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:08.126976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:08.126976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:08.128971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:08.128971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:08.128971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:08.128971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:08.128971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 324/673] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:08.131061 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f88bc3-e84d-4812-802e-f1a6485cd067 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:08.157156 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ec6de-72e9-4e2d-8ec1-9ed9221ea313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:08.157959 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ec6de-72e9-4e2d-8ec1-9ed9221ea313 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:08.158309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ec6de-72e9-4e2d-8ec1-9ed9221ea313 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:08.158661 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ec6de-72e9-4e2d-8ec1-9ed9221ea313 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:08.159528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ec6de-72e9-4e2d-8ec1-9ed9221ea313 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:08.159897 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ec6de-72e9-4e2d-8ec1-9ed9221ea313 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:08.162048 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ec6de-72e9-4e2d-8ec1-9ed9221ea313 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:08.182185 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 350/674] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:08.183800 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ec6de-72e9-4e2d-8ec1-9ed9221ea313 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:08.271407 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f88bc3-e84d-4812-802e-f1a6485cd067 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:08.298914 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:08.311526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb0d62-30e1-4fa6-8a55-c937c503dd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:08.312317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb0d62-30e1-4fa6-8a55-c937c503dd54 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:08.312654 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb0d62-30e1-4fa6-8a55-c937c503dd54 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:08.312993 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb0d62-30e1-4fa6-8a55-c937c503dd54 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:08.314304 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb0d62-30e1-4fa6-8a55-c937c503dd54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:08.314304 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb0d62-30e1-4fa6-8a55-c937c503dd54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:08.315749 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb0d62-30e1-4fa6-8a55-c937c503dd54 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:08.334922 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 325/675] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:08.334922 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb0d62-30e1-4fa6-8a55-c937c503dd54 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:08.360883 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:08.379146 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ec6de-72e9-4e2d-8ec1-9ed9221ea313 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:08.457768 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e4443-821f-4656-adc8-761b057b0b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:08.459629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e4443-821f-4656-adc8-761b057b0b02 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:08.461152 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e4443-821f-4656-adc8-761b057b0b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:08.462251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e4443-821f-4656-adc8-761b057b0b02 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:08.463660 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e4443-821f-4656-adc8-761b057b0b02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:08.464519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e4443-821f-4656-adc8-761b057b0b02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:08.467461 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e4443-821f-4656-adc8-761b057b0b02 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:08.493307 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 351/676] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:08.495283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e4443-821f-4656-adc8-761b057b0b02 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:08.511860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb0d62-30e1-4fa6-8a55-c937c503dd54 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:08.570536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557085e-1688-46f0-a81c-477579115219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:08.571398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557085e-1688-46f0-a81c-477579115219 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:08.571731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557085e-1688-46f0-a81c-477579115219 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:08.572060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557085e-1688-46f0-a81c-477579115219 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:08.580727 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557085e-1688-46f0-a81c-477579115219 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:08.582253 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557085e-1688-46f0-a81c-477579115219 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:08.601203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-0557085e-1688-46f0-a81c-477579115219 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 16 14:50:08.601203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:08.601203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:08.601203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:08.601203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:08.601203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:08.601203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:08.601203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:08.601203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:08.601203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:08.601203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:08.601203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:08.601203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:50:08.601203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:50:08.601203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:50:08.601203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:50:08.602657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:08.602657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:08.602657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:50:08.602657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:08.602657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:50:08.602657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:08.602657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:08.602657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:08.602657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:08.602657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:08.602657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:08.602657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:08.602657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:08.602657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:08.602657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:08.602657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:08.602657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:08.602657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:08.604271 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:08.604271 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:08.604271 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:50:08.604271 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:50:08.604271 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:50:08.604271 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:50:08.604271 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:50:08.604271 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:50:08.604271 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 16 14:50:08.604271 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:08.604271 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 326/677] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:50:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:08.604873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557085e-1688-46f0-a81c-477579115219 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:08.652643 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e4443-821f-4656-adc8-761b057b0b02 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:08.662521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:08.695778 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e860a8-015b-4640-8660-1dde51baa825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:08.696585 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e860a8-015b-4640-8660-1dde51baa825 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:08.696859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e860a8-015b-4640-8660-1dde51baa825 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:08.697128 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e860a8-015b-4640-8660-1dde51baa825 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:08.704582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e860a8-015b-4640-8660-1dde51baa825 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:08.706022 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e860a8-015b-4640-8660-1dde51baa825 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:08.723485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-f3e860a8-015b-4640-8660-1dde51baa825 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:08.723485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:08.723485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:08.723485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:08.723485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:08.723485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:08.723485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:08.723485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:08.723485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:08.723485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:08.723485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:08.723485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:08.723485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:50:08.723485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:50:08.723485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:50:08.724472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:50:08.724472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:08.724472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:08.724472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:08.724472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:08.724472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:08.724472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:08.724472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:08.724472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:08.724472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:08.724472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:08.724472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:08.724472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:08.724472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:08.724472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:08.724472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:08.724472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:08.725368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:50:08.725368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:50:08.725368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:50:08.725368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:50:08.725368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:50:08.725368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:50:08.725368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:08.725368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:08.725368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 352/678] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:08.726372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d0031-504a-4c1a-8a20-ec49a9dc61d0 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:08.726372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d0031-504a-4c1a-8a20-ec49a9dc61d0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:08.726656 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d0031-504a-4c1a-8a20-ec49a9dc61d0 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:08.728203 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 353/679] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:08.728721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d0031-504a-4c1a-8a20-ec49a9dc61d0 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:08.737193 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557085e-1688-46f0-a81c-477579115219 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:08.754410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:08.780598 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f02f39-c865-442c-9ea6-235d03e30d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:08.781720 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f02f39-c865-442c-9ea6-235d03e30d59 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:08.782103 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f02f39-c865-442c-9ea6-235d03e30d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:08.782452 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f02f39-c865-442c-9ea6-235d03e30d59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:08.791719 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f02f39-c865-442c-9ea6-235d03e30d59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:08.793386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f02f39-c865-442c-9ea6-235d03e30d59 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:08.812696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-f3f02f39-c865-442c-9ea6-235d03e30d59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:08.812696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:08.812696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:08.812696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:08.812696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:08.812696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:08.812696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:08.812696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:08.812696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:08.812696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:08.812696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:08.812696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:08.812696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:50:08.812696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:50:08.812696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:50:08.813729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:50:08.813729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:08.813729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:08.813729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:08.813729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:08.813729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:08.813729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:08.813729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:08.813729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:08.813729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:08.813729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:08.813729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:08.813729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:08.813729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:08.813729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:08.813729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:08.813729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:08.821663 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:50:08.821663 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:50:08.821663 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:50:08.821663 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:50:08.821663 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:50:08.821663 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:50:08.821663 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:08.821663 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:08.821663 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 327/680] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:08.821663 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f02f39-c865-442c-9ea6-235d03e30d59 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:08.906484 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d0031-504a-4c1a-8a20-ec49a9dc61d0 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:08.969992 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdd261-625e-420b-8bce-1e081ad5543d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:08.970265 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f02f39-c865-442c-9ea6-235d03e30d59 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:08.970740 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdd261-625e-420b-8bce-1e081ad5543d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:08.971243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdd261-625e-420b-8bce-1e081ad5543d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:08.971854 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdd261-625e-420b-8bce-1e081ad5543d None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:08.972779 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bdd261-625e-420b-8bce-1e081ad5543d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:08.974242 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bdd261-625e-420b-8bce-1e081ad5543d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:08.974710 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bdd261-625e-420b-8bce-1e081ad5543d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:08.975569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:09.012037 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 354/681] 10.222.0.27 () {56 vars in 1135 bytes} [Sat Jul 16 14:50:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:09.012037 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63faf7-3063-4156-ba7f-2dac7fcf96f2 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:09.012037 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63faf7-3063-4156-ba7f-2dac7fcf96f2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:09.012037 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63faf7-3063-4156-ba7f-2dac7fcf96f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:09.019022 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b19c7-82c1-4707-9a3f-3aa45bce2534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:09.019827 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b19c7-82c1-4707-9a3f-3aa45bce2534 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:09.019978 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b19c7-82c1-4707-9a3f-3aa45bce2534 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:09.020325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b19c7-82c1-4707-9a3f-3aa45bce2534 None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:09.020992 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b19c7-82c1-4707-9a3f-3aa45bce2534 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:09.021295 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b19c7-82c1-4707-9a3f-3aa45bce2534 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:09.022927 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b19c7-82c1-4707-9a3f-3aa45bce2534 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:09.037516 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 328/682] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:50:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:09.038721 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b19c7-82c1-4707-9a3f-3aa45bce2534 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:09.044831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5e63faf7-3063-4156-ba7f-2dac7fcf96f2 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:09.047069 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5e63faf7-3063-4156-ba7f-2dac7fcf96f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:09.144188 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:09.162087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 355/683] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:50:09.162087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63faf7-3063-4156-ba7f-2dac7fcf96f2 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:09.194317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b19c7-82c1-4707-9a3f-3aa45bce2534 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:09.248583 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273bddc-c1d2-4449-b4b0-ab59fd336f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:09.250208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273bddc-c1d2-4449-b4b0-ab59fd336f37 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:09.250970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273bddc-c1d2-4449-b4b0-ab59fd336f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:09.251481 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273bddc-c1d2-4449-b4b0-ab59fd336f37 None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:09.253102 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273bddc-c1d2-4449-b4b0-ab59fd336f37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:09.253650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273bddc-c1d2-4449-b4b0-ab59fd336f37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:09.256520 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273bddc-c1d2-4449-b4b0-ab59fd336f37 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:09.282935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 329/684] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:50:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:09.283989 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273bddc-c1d2-4449-b4b0-ab59fd336f37 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:09.324841 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63faf7-3063-4156-ba7f-2dac7fcf96f2 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:09.337822 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:09.384898 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab66f63-60fe-4792-9318-90378554a32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:09.386054 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab66f63-60fe-4792-9318-90378554a32d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:09.386361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab66f63-60fe-4792-9318-90378554a32d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:09.386640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab66f63-60fe-4792-9318-90378554a32d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:09.467165 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273bddc-c1d2-4449-b4b0-ab59fd336f37 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:09.501066 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:09.505591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade4109-df24-4b2c-b351-612d27ec6091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:09.506267 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade4109-df24-4b2c-b351-612d27ec6091 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:09.506536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade4109-df24-4b2c-b351-612d27ec6091 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:09.506816 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade4109-df24-4b2c-b351-612d27ec6091 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:09.508455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade4109-df24-4b2c-b351-612d27ec6091 None admin] RBAC: Adding query filter params (role.id=21119ebd056740048b9ec639ebb20446, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=6e372cc62af24664b01a967435eb8902) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:09.508634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab66f63-60fe-4792-9318-90378554a32d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:09.509092 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade4109-df24-4b2c-b351-612d27ec6091 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:09.510492 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade4109-df24-4b2c-b351-612d27ec6091 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:09.510940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab66f63-60fe-4792-9318-90378554a32d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:09.517067 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 330/685] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:50:09 2022] GET /identity/v3/role_assignments?user.id=6e372cc62af24664b01a967435eb8902&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=21119ebd056740048b9ec639ebb20446 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:09.522071 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade4109-df24-4b2c-b351-612d27ec6091 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:09.659591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:09.676980 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade4109-df24-4b2c-b351-612d27ec6091 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:09.724741 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 356/686] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:50:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:50:09.725852 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab66f63-60fe-4792-9318-90378554a32d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:09.756156 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0169d5-d173-474e-a405-cf1b9f049fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:09.759440 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0169d5-d173-474e-a405-cf1b9f049fab None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:09.760546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0169d5-d173-474e-a405-cf1b9f049fab None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:09.760546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0169d5-d173-474e-a405-cf1b9f049fab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:09.777003 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0169d5-d173-474e-a405-cf1b9f049fab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:50:09.784973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-8d0169d5-d173-474e-a405-cf1b9f049fab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:50:09.784973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:09.784973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:09.784973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:09.784973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:09.784973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:09.784973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:09.784973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:09.784973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:09.784973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:09.784973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:09.784973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:09.784973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:50:09.784973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:50:09.784973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:50:09.784973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:50:09.786015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:50:09.786015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:50:09.786015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:50:09.786015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:50:09.786015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:09.786015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:09.786015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:09.786015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:09.786015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:09.786015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:09.786015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:09.786015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:09.786015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:09.786015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:09.786015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:09.786015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:09.786961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:09.786961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:09.786961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:09.786961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:09.786961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:50:09.786961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:50:09.786961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:50:09.786961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:50:09.786961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:50:09.786961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:50:09.786961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:50:09.786961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:09.786961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 331/687] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:50:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:09.788007 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62742764-d37d-44b3-89da-c0c351c71cb0 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:09.788099 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62742764-d37d-44b3-89da-c0c351c71cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:09.788321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62742764-d37d-44b3-89da-c0c351c71cb0 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:09.789899 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 332/688] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:09.790563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62742764-d37d-44b3-89da-c0c351c71cb0 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:09.865177 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:09.869674 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab66f63-60fe-4792-9318-90378554a32d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:09.925094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e66a9-143f-4bf1-81c7-2741bf7a1da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:09.926217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e66a9-143f-4bf1-81c7-2741bf7a1da4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:09.926701 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e66a9-143f-4bf1-81c7-2741bf7a1da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:09.927205 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e66a9-143f-4bf1-81c7-2741bf7a1da4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:09.929095 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e66a9-143f-4bf1-81c7-2741bf7a1da4 None admin] RBAC: Adding query filter params (role.id=21119ebd056740048b9ec639ebb20446, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=8c21d12c977e4c36b4f0ed442b11ef49) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:09.929769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e66a9-143f-4bf1-81c7-2741bf7a1da4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:09.932633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e66a9-143f-4bf1-81c7-2741bf7a1da4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:09.944523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 357/689] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:50:09 2022] GET /identity/v3/role_assignments?user.id=8c21d12c977e4c36b4f0ed442b11ef49&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=21119ebd056740048b9ec639ebb20446 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:09.947088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd573421-ba67-4935-b8ee-702ad31ab1ce None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:09.947605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd573421-ba67-4935-b8ee-702ad31ab1ce None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:09.948109 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd573421-ba67-4935-b8ee-702ad31ab1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:09.972263 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-cd573421-ba67-4935-b8ee-702ad31ab1ce None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:09.973798 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-cd573421-ba67-4935-b8ee-702ad31ab1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:09.983799 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62742764-d37d-44b3-89da-c0c351c71cb0 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:10.055217 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d87690-6313-4d6c-a955-9f300c6346df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:10.056649 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d87690-6313-4d6c-a955-9f300c6346df None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:10.057150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d87690-6313-4d6c-a955-9f300c6346df None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:10.057788 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d87690-6313-4d6c-a955-9f300c6346df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:10.078635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d87690-6313-4d6c-a955-9f300c6346df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:10.080580 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d87690-6313-4d6c-a955-9f300c6346df None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:10.092253 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:10.096780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 358/690] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:50:10.097992 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd573421-ba67-4935-b8ee-702ad31ab1ce None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:10.104369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-83d87690-6313-4d6c-a955-9f300c6346df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:10.104369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:10.104369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:10.104369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:10.104369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:10.104369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:10.104369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:10.104369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:10.104369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:10.104369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:10.104369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:10.104369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:10.104369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:10.104369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:10.104369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:10.123464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:10.123464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:10.123464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:10.123464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:10.123464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:10.123464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:10.123464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:10.123464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:10.123464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:10.123464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:10.123464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:10.123464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:10.123464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:10.123464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:10.123464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:10.123464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:10.123464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:10.124365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:10.124365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:10.124365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:10.124365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:10.124365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 333/691] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:10.124365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d87690-6313-4d6c-a955-9f300c6346df None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:10.216613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:10.228535 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd573421-ba67-4935-b8ee-702ad31ab1ce None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:10.242814 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d87690-6313-4d6c-a955-9f300c6346df None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:10.273772 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35904f08-a593-4406-bd76-2f7d1be9070b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:10.274466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35904f08-a593-4406-bd76-2f7d1be9070b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:10.274748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35904f08-a593-4406-bd76-2f7d1be9070b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:10.275069 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35904f08-a593-4406-bd76-2f7d1be9070b None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:10.275898 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35904f08-a593-4406-bd76-2f7d1be9070b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:10.276208 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35904f08-a593-4406-bd76-2f7d1be9070b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:10.277629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35904f08-a593-4406-bd76-2f7d1be9070b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:10.286028 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d873c-5b03-4f03-acb3-080c67a91ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:10.286988 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d873c-5b03-4f03-acb3-080c67a91ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:10.287427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d873c-5b03-4f03-acb3-080c67a91ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:10.287939 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d873c-5b03-4f03-acb3-080c67a91ed8 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:10.288172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 359/692] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:50:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:10.289850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35904f08-a593-4406-bd76-2f7d1be9070b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:10.290413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d873c-5b03-4f03-acb3-080c67a91ed8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:10.290988 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d873c-5b03-4f03-acb3-080c67a91ed8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:10.292678 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521d873c-5b03-4f03-acb3-080c67a91ed8 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:10.308538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 334/693] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:10.309750 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d873c-5b03-4f03-acb3-080c67a91ed8 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:10.398408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:10.448141 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d873c-5b03-4f03-acb3-080c67a91ed8 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:10.459088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35904f08-a593-4406-bd76-2f7d1be9070b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:10.496666 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c032f6-d587-4cf0-96cf-dd53a851141d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:10.498143 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c032f6-d587-4cf0-96cf-dd53a851141d None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:10.498732 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c032f6-d587-4cf0-96cf-dd53a851141d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:10.499307 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c032f6-d587-4cf0-96cf-dd53a851141d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:10.505502 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2eff8-310e-4e4c-858b-b7b63e23b978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:10.506330 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2eff8-310e-4e4c-858b-b7b63e23b978 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:10.506611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2eff8-310e-4e4c-858b-b7b63e23b978 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:10.506913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b2eff8-310e-4e4c-858b-b7b63e23b978 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:10.518095 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c032f6-d587-4cf0-96cf-dd53a851141d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:10.520806 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c032f6-d587-4cf0-96cf-dd53a851141d None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:10.521251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2eff8-310e-4e4c-858b-b7b63e23b978 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:10.523279 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b2eff8-310e-4e4c-858b-b7b63e23b978 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:10.549235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-17c032f6-d587-4cf0-96cf-dd53a851141d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:10.549235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:10.549235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:10.549235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:10.549235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:10.549235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:10.549235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:10.549235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:10.549235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:10.549235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:10.549235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:10.549235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:10.549235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:10.549235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:10.549235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:10.551032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:10.551032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:10.551032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:10.551032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:10.551032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:10.551032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:10.551032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:10.551032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:10.551032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:10.551032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:10.551032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:10.551032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:10.551032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:10.551032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:10.551032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:10.551032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:10.551032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:10.552684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:10.552684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:10.552684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:10.552684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:10.552684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 335/694] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:10.552684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c032f6-d587-4cf0-96cf-dd53a851141d None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:10.557079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-f7b2eff8-310e-4e4c-858b-b7b63e23b978 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:10.557079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:10.557079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:10.557079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:10.557079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:10.557079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:10.557079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:10.557079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:10.557079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:10.557079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:10.557079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:10.557079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:10.557079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:10.557079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:10.557079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:10.558950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:10.558950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:10.558950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:10.558950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:10.558950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:10.558950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:10.558950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:10.558950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:10.558950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:10.558950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:10.558950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:10.558950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:10.558950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:10.558950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:10.558950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:10.558950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:10.558950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:10.560901 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:10.560901 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:10.560901 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:10.560901 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:10.566948 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 360/695] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:10.580479 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2eff8-310e-4e4c-858b-b7b63e23b978 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:10.600388 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:10.695603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c032f6-d587-4cf0-96cf-dd53a851141d None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:10.716866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:10.744544 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0267ef8-f571-4b7a-b68f-9c672f5b4cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:10.745579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0267ef8-f571-4b7a-b68f-9c672f5b4cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:10.745925 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0267ef8-f571-4b7a-b68f-9c672f5b4cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:10.746262 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0267ef8-f571-4b7a-b68f-9c672f5b4cd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:10.754935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0267ef8-f571-4b7a-b68f-9c672f5b4cd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:10.758010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0267ef8-f571-4b7a-b68f-9c672f5b4cd2 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:10.776887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-b0267ef8-f571-4b7a-b68f-9c672f5b4cd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:10.776887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:10.776887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:10.776887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:10.776887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:10.776887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:10.776887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:10.776887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:10.776887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:10.776887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:10.776887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:10.776887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:10.776887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:50:10.776887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:50:10.776887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:50:10.778686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:50:10.778686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:10.778686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:10.778686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:10.778686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:10.778686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:10.778686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:10.778686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:10.778686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:10.778686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:10.778686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:10.778686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:10.778686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:10.778686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:10.778686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:10.778686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:10.778686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:10.780521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:50:10.780521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:50:10.780521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:50:10.780521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:50:10.780521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:50:10.780521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:50:10.780521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:10.780521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:10.783065 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 336/696] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:10.784036 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0267ef8-f571-4b7a-b68f-9c672f5b4cd2 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:10.812555 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b2eff8-310e-4e4c-858b-b7b63e23b978 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:10.884925 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b49e8-e46c-4ff1-93cd-c61c86ac0d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:10.886589 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b49e8-e46c-4ff1-93cd-c61c86ac0d6a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:10.887351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b49e8-e46c-4ff1-93cd-c61c86ac0d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:10.888080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40b49e8-e46c-4ff1-93cd-c61c86ac0d6a None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:10.889610 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b49e8-e46c-4ff1-93cd-c61c86ac0d6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:10.890553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b49e8-e46c-4ff1-93cd-c61c86ac0d6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:10.893427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40b49e8-e46c-4ff1-93cd-c61c86ac0d6a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:10.923511 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 361/697] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:10.925206 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b49e8-e46c-4ff1-93cd-c61c86ac0d6a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:10.933276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0267ef8-f571-4b7a-b68f-9c672f5b4cd2 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:10.981926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcff8f-4dff-43c9-b30b-e3aac1ae4a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:10.983007 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcff8f-4dff-43c9-b30b-e3aac1ae4a17 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:10.983453 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcff8f-4dff-43c9-b30b-e3aac1ae4a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:10.983862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcff8f-4dff-43c9-b30b-e3aac1ae4a17 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:10.985029 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdcff8f-4dff-43c9-b30b-e3aac1ae4a17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:10.985488 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdcff8f-4dff-43c9-b30b-e3aac1ae4a17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:10.987921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdcff8f-4dff-43c9-b30b-e3aac1ae4a17 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:11.018918 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 337/698] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:11.020329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcff8f-4dff-43c9-b30b-e3aac1ae4a17 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:11.047496 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:11.076750 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40b49e8-e46c-4ff1-93cd-c61c86ac0d6a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:11.091511 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:11.119449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd74c6-f6e2-4e6c-be05-80aca6e4489d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:11.120479 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd74c6-f6e2-4e6c-be05-80aca6e4489d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:11.120928 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd74c6-f6e2-4e6c-be05-80aca6e4489d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:11.121365 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd74c6-f6e2-4e6c-be05-80aca6e4489d None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:11.122488 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd74c6-f6e2-4e6c-be05-80aca6e4489d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:11.123036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd74c6-f6e2-4e6c-be05-80aca6e4489d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:11.124714 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd74c6-f6e2-4e6c-be05-80aca6e4489d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:11.143822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 362/699] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:50:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:11.145103 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd74c6-f6e2-4e6c-be05-80aca6e4489d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:11.220760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcff8f-4dff-43c9-b30b-e3aac1ae4a17 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:11.258922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd74c6-f6e2-4e6c-be05-80aca6e4489d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:11.288089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17b3d4-e367-4db5-9028-9f59e3ba7eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:11.289936 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17b3d4-e367-4db5-9028-9f59e3ba7eef None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:11.290433 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17b3d4-e367-4db5-9028-9f59e3ba7eef None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:11.290945 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17b3d4-e367-4db5-9028-9f59e3ba7eef None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:11.292353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17b3d4-e367-4db5-9028-9f59e3ba7eef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:11.295161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17b3d4-e367-4db5-9028-9f59e3ba7eef None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:11.295402 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:11.300616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12b1a8-bfeb-48c7-bf18-3595a732065b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:11.301763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12b1a8-bfeb-48c7-bf18-3595a732065b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:11.302264 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12b1a8-bfeb-48c7-bf18-3595a732065b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:11.302738 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12b1a8-bfeb-48c7-bf18-3595a732065b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:11.313171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d12b1a8-bfeb-48c7-bf18-3595a732065b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:11.315203 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d12b1a8-bfeb-48c7-bf18-3595a732065b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:11.331498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-6d12b1a8-bfeb-48c7-bf18-3595a732065b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 16 14:50:11.331498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:11.331498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:11.331498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:11.331498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:11.331498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:11.331498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:11.331498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:11.331498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:11.331498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:11.331498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:11.331498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:11.331498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:50:11.331498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:50:11.331498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:50:11.331498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:50:11.333306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:11.333306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:11.333306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:50:11.333306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:11.333306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:50:11.333306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:11.333306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:11.333306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:11.333306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:11.333306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:11.333306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:11.333306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:11.333306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:11.333306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:11.333306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:11.333306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:11.333306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:11.333306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:11.336283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:11.336283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:11.336283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:50:11.336283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:50:11.336283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:50:11.336283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:50:11.336283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:50:11.336283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:50:11.336283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 16 14:50:11.336283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:11.338969 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 363/700] 10.222.0.27 () {56 vars in 1093 bytes} [Sat Jul 16 14:50:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:11.338969 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12b1a8-bfeb-48c7-bf18-3595a732065b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:11.407643 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 338/701] 10.222.0.27 () {60 vars in 1132 bytes} [Sat Jul 16 14:50:11 2022] POST /identity/v3/users => generated 254 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:50:11.408603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17b3d4-e367-4db5-9028-9f59e3ba7eef None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:11.542922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12b1a8-bfeb-48c7-bf18-3595a732065b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:11.628190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df47a4-0416-4a58-ad30-143ae97531d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:11.630189 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df47a4-0416-4a58-ad30-143ae97531d9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:11.631021 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df47a4-0416-4a58-ad30-143ae97531d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:11.631780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df47a4-0416-4a58-ad30-143ae97531d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:11.634713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df47a4-0416-4a58-ad30-143ae97531d9 None admin] RBAC: Adding query filter params (role.id=21119ebd056740048b9ec639ebb20446, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=e1464c83fb294a3d90e5763ef648e834) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:11.635659 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df47a4-0416-4a58-ad30-143ae97531d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:11.640342 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df47a4-0416-4a58-ad30-143ae97531d9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:11.657416 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 364/702] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:50:11 2022] GET /identity/v3/role_assignments?user.id=e1464c83fb294a3d90e5763ef648e834&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=21119ebd056740048b9ec639ebb20446 => generated 608 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:11.669820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:11.685101 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17b3d4-e367-4db5-9028-9f59e3ba7eef None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:11.747040 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103706d-1b31-48df-88f0-7766171510ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:11.747758 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103706d-1b31-48df-88f0-7766171510ca None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:11.748043 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103706d-1b31-48df-88f0-7766171510ca None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:11.748316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103706d-1b31-48df-88f0-7766171510ca None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:11.749226 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103706d-1b31-48df-88f0-7766171510ca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:11.749536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103706d-1b31-48df-88f0-7766171510ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:11.751267 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103706d-1b31-48df-88f0-7766171510ca None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:11.780652 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 339/703] 10.222.0.27 () {56 vars in 1133 bytes} [Sat Jul 16 14:50:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:11.787183 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df47a4-0416-4a58-ad30-143ae97531d9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:11.875351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:11.946691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df47a4-0416-4a58-ad30-143ae97531d9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:11.989443 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668918e5-2750-4b4b-a566-e2ed507b9538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:11.990204 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668918e5-2750-4b4b-a566-e2ed507b9538 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:11.990485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668918e5-2750-4b4b-a566-e2ed507b9538 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:11.990758 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668918e5-2750-4b4b-a566-e2ed507b9538 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:12.131594 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668918e5-2750-4b4b-a566-e2ed507b9538 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:12.134703 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668918e5-2750-4b4b-a566-e2ed507b9538 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:12.237334 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:12.312697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 365/704] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:50:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:50:12.321311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103706d-1b31-48df-88f0-7766171510ca None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:12.549858 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103706d-1b31-48df-88f0-7766171510ca None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:12.602720 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668ba77-5020-423d-bd8f-3f2cf7889f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:12.603918 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668ba77-5020-423d-bd8f-3f2cf7889f9f None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:12.604413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668ba77-5020-423d-bd8f-3f2cf7889f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:12.604912 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668ba77-5020-423d-bd8f-3f2cf7889f9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:12.622212 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668ba77-5020-423d-bd8f-3f2cf7889f9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:12.624082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668ba77-5020-423d-bd8f-3f2cf7889f9f None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:12.635840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912da77-5a19-4525-b983-9bc18b620720 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:12.636890 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912da77-5a19-4525-b983-9bc18b620720 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:12.637397 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912da77-5a19-4525-b983-9bc18b620720 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:12.640422 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 366/705] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:12.646113 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc504bf-6733-4149-a0af-937af05a2fcb None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:12.646607 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc504bf-6733-4149-a0af-937af05a2fcb None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:12.647108 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc504bf-6733-4149-a0af-937af05a2fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:12.651148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:12.671565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-0668ba77-5020-423d-bd8f-3f2cf7889f9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:12.671565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:12.671565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:12.671565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:12.671565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:12.671565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:12.671565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:12.671565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:12.671565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:12.671565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:12.671565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:12.671565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:12.671565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:12.671565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:12.671565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:12.673232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:12.673232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:12.673232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:12.673232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:12.673232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:12.673232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:12.673232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:12.673232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:12.673232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:12.673232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:12.673232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:12.673232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:12.673232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:12.673232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:12.673232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:12.673232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:12.673232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:12.674876 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:12.674876 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:12.674876 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:12.674876 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:12.674876 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 340/706] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:12.684598 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668ba77-5020-423d-bd8f-3f2cf7889f9f None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:12.704814 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-2cc504bf-6733-4149-a0af-937af05a2fcb None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:12.706945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2cc504bf-6733-4149-a0af-937af05a2fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:12.888924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 367/707] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:50:12.912191 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc504bf-6733-4149-a0af-937af05a2fcb None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:13.018765 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668ba77-5020-423d-bd8f-3f2cf7889f9f None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:13.129821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77683f-6ce7-4578-ac05-62f39f6ba6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:13.131448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77683f-6ce7-4578-ac05-62f39f6ba6bf None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:13.132911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77683f-6ce7-4578-ac05-62f39f6ba6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:13.139559 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77683f-6ce7-4578-ac05-62f39f6ba6bf None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:13.140287 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77683f-6ce7-4578-ac05-62f39f6ba6bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:13.140609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77683f-6ce7-4578-ac05-62f39f6ba6bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:13.142036 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77683f-6ce7-4578-ac05-62f39f6ba6bf None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:13.175583 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 341/708] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 493 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:13.181279 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77683f-6ce7-4578-ac05-62f39f6ba6bf None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:13.186096 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:13.264722 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc504bf-6733-4149-a0af-937af05a2fcb None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:13.379597 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:13.385501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e00e8-9990-40af-a701-75e5ac76337f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:13.387329 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e00e8-9990-40af-a701-75e5ac76337f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:13.397283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e00e8-9990-40af-a701-75e5ac76337f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:13.397283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e00e8-9990-40af-a701-75e5ac76337f None admin] PATH_INFO: `/v3/services/network` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:13.397283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e00e8-9990-40af-a701-75e5ac76337f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:13.397283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e00e8-9990-40af-a701-75e5ac76337f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:13.427847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-f30e00e8-9990-40af-a701-75e5ac76337f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 16 14:50:13.427847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:13.427847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:13.427847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:13.427847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:13.427847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:13.427847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:13.427847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:13.427847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 16 14:50:13.427847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:13.427847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:13.427847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:13.427847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:13.427847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:13.427847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:13.427847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:13.427847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:13.430440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:13.430440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:13.430440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:13.430440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:13.430440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:13.430440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:13.430440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:13.430440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:13.430440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:13.430440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:13.430440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:13.430440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:13.430440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:13.430440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:13.430440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:13.430440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:13.432070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:13.432070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:13.432070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:13.432070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:13.432070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:13.432070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:13.432070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:13.432070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:13.432070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:13.432070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:13.432070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:13.432070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:13.432070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 16 14:50:13.432070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:13.437370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 368/709] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:12 2022] GET /identity/v3/services/network => generated 88 bytes in 526 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:13.446625 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e00e8-9990-40af-a701-75e5ac76337f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:13.477367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77683f-6ce7-4578-ac05-62f39f6ba6bf None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:13.553679 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1c4ea-7d28-439b-b6b2-4f06e11cafc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:13.556306 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1c4ea-7d28-439b-b6b2-4f06e11cafc4 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:13.556611 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1c4ea-7d28-439b-b6b2-4f06e11cafc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:13.556866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1c4ea-7d28-439b-b6b2-4f06e11cafc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:13.568508 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df1c4ea-7d28-439b-b6b2-4f06e11cafc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:13.570062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df1c4ea-7d28-439b-b6b2-4f06e11cafc4 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:13.620197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-5df1c4ea-7d28-439b-b6b2-4f06e11cafc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:13.620197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:13.620197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:13.620197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:13.620197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:13.620197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:13.620197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:13.620197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:13.620197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:13.620197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:13.620197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:13.620197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:13.620197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:50:13.620197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:50:13.620197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:50:13.621148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:50:13.621148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:13.621148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:13.621148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:13.621148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:13.621148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:13.621148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:13.621148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:13.621148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:13.621148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:13.621148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:13.621148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:13.621148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:13.621148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:13.621148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:13.621148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:13.621148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:13.622053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:50:13.622053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:50:13.622053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:50:13.622053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:50:13.622053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:50:13.622053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:50:13.622053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:13.622053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:13.624940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 342/710] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 445 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:13.632775 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1c4ea-7d28-439b-b6b2-4f06e11cafc4 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:13.713846 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e00e8-9990-40af-a701-75e5ac76337f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:13.775269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6cc2c4-e052-4314-a7a6-98b035c2a1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:13.776399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6cc2c4-e052-4314-a7a6-98b035c2a1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:13.776879 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6cc2c4-e052-4314-a7a6-98b035c2a1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:13.777353 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6cc2c4-e052-4314-a7a6-98b035c2a1b1 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:13.778630 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6cc2c4-e052-4314-a7a6-98b035c2a1b1 None admin] RBAC: Adding query filter params (name=network) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:13.779208 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6cc2c4-e052-4314-a7a6-98b035c2a1b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:13.782334 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6cc2c4-e052-4314-a7a6-98b035c2a1b1 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:13.804851 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 369/711] 10.222.0.27 () {56 vars in 1112 bytes} [Sat Jul 16 14:50:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:13.810896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6cc2c4-e052-4314-a7a6-98b035c2a1b1 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:13.846803 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:13.928738 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df1c4ea-7d28-439b-b6b2-4f06e11cafc4 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:13.987561 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:14.024758 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd488b-d32a-461b-9d1b-797bbcb1257b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:14.025895 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd488b-d32a-461b-9d1b-797bbcb1257b None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:14.026487 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd488b-d32a-461b-9d1b-797bbcb1257b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:14.027035 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd488b-d32a-461b-9d1b-797bbcb1257b None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:14.028578 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd488b-d32a-461b-9d1b-797bbcb1257b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:14.029225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd488b-d32a-461b-9d1b-797bbcb1257b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:14.032098 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd488b-d32a-461b-9d1b-797bbcb1257b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:14.032376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6cc2c4-e052-4314-a7a6-98b035c2a1b1 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:14.063406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 343/712] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:50:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:14.066034 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c7a3e-af51-41da-a755-e83159ac6c07 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:14.066549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c7a3e-af51-41da-a755-e83159ac6c07 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:14.067068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c7a3e-af51-41da-a755-e83159ac6c07 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:14.070477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f817b9e-6066-4127-8f85-ea3de5492226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:14.071265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f817b9e-6066-4127-8f85-ea3de5492226 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:14.071542 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f817b9e-6066-4127-8f85-ea3de5492226 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:14.071810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f817b9e-6066-4127-8f85-ea3de5492226 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:14.072537 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f817b9e-6066-4127-8f85-ea3de5492226 None admin] RBAC: Adding query filter params (type=network) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:14.072810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f817b9e-6066-4127-8f85-ea3de5492226 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:14.074205 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f817b9e-6066-4127-8f85-ea3de5492226 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:14.075294 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 344/713] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:14.079291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c7a3e-af51-41da-a755-e83159ac6c07 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:14.080986 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 370/714] 10.222.0.27 () {56 vars in 1112 bytes} [Sat Jul 16 14:50:13 2022] GET /identity/v3/services?type=network => generated 124 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:14.082279 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c431ac1-e2e0-4126-875c-935978e2a0d1 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:14.082552 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c431ac1-e2e0-4126-875c-935978e2a0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:14.082829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c431ac1-e2e0-4126-875c-935978e2a0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:14.119981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-8c431ac1-e2e0-4126-875c-935978e2a0d1 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:14.122218 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-8c431ac1-e2e0-4126-875c-935978e2a0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:14.233713 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c7a3e-af51-41da-a755-e83159ac6c07 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:14.323108 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a3508-b3f8-42d8-864e-4a265ba199c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:14.323686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a3508-b3f8-42d8-864e-4a265ba199c4 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:14.323963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a3508-b3f8-42d8-864e-4a265ba199c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:14.324246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a3508-b3f8-42d8-864e-4a265ba199c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:14.325089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a3508-b3f8-42d8-864e-4a265ba199c4 None admin] RBAC: Adding query filter params (role.id=0efeff46573247f3b9f1e2dec2968672, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=3f2e241494244edf8d283e8f44192c46) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:14.325398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a3508-b3f8-42d8-864e-4a265ba199c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:14.326933 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a3508-b3f8-42d8-864e-4a265ba199c4 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:14.330925 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 371/715] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 16 14:50:14.337417 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 345/716] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:50:14 2022] GET /identity/v3/role_assignments?user.id=3f2e241494244edf8d283e8f44192c46&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=0efeff46573247f3b9f1e2dec2968672 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:14.388222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c431ac1-e2e0-4126-875c-935978e2a0d1 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:14.390731 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:14.516451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c431ac1-e2e0-4126-875c-935978e2a0d1 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:14.573461 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad94ed-84c1-4c21-b85a-b5ea01ff0bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:14.574728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad94ed-84c1-4c21-b85a-b5ea01ff0bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:14.575227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad94ed-84c1-4c21-b85a-b5ea01ff0bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:14.575685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad94ed-84c1-4c21-b85a-b5ea01ff0bd3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:14.576876 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad94ed-84c1-4c21-b85a-b5ea01ff0bd3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:14.583686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad94ed-84c1-4c21-b85a-b5ea01ff0bd3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:14.609246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-c6ad94ed-84c1-4c21-b85a-b5ea01ff0bd3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 16 14:50:14.609246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:14.609246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:14.609246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:14.609246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:14.609246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:14.609246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:14.609246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:14.609246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 16 14:50:14.609246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:14.609246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:14.609246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:14.609246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:14.609246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:14.609246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:14.609246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:14.611076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:14.611076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:14.611076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:14.611076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:14.611076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:14.611076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:14.611076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:14.611076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:14.611076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:14.611076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:14.611076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:14.611076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:14.611076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:14.611076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:14.611076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:14.611076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:14.612715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:14.612715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:14.612715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:14.612715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:14.612715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:14.612715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:14.612715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:14.612715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:14.612715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:14.612715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:14.612715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:14.612715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:14.612715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:14.612715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 16 14:50:14.612715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:14.612715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 372/717] 10.222.0.27 () {56 vars in 1129 bytes} [Sat Jul 16 14:50:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:14.625651 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a3508-b3f8-42d8-864e-4a265ba199c4 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:14.721167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:14.838007 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a3508-b3f8-42d8-864e-4a265ba199c4 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:14.901649 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471297c-b95a-477a-a04c-c3ac91ecf7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:14.902693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471297c-b95a-477a-a04c-c3ac91ecf7bf None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:14.903422 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471297c-b95a-477a-a04c-c3ac91ecf7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:14.903906 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471297c-b95a-477a-a04c-c3ac91ecf7bf None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:14.904879 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471297c-b95a-477a-a04c-c3ac91ecf7bf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:14.905409 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471297c-b95a-477a-a04c-c3ac91ecf7bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:14.907551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0471297c-b95a-477a-a04c-c3ac91ecf7bf None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:14.912879 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 346/718] 10.222.0.27 () {56 vars in 1124 bytes} [Sat Jul 16 14:50:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:14.915916 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ab0bf-40dc-4776-80f1-4b68810f7616 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:14.916761 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ab0bf-40dc-4776-80f1-4b68810f7616 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:14.917599 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ab0bf-40dc-4776-80f1-4b68810f7616 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:14.919403 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471297c-b95a-477a-a04c-c3ac91ecf7bf None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:14.921403 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 373/719] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:14.930943 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d97f0-181e-488e-acec-6e4802c1adcd None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:14.931877 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d97f0-181e-488e-acec-6e4802c1adcd None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:14.932604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d97f0-181e-488e-acec-6e4802c1adcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:14.983267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b02d97f0-181e-488e-acec-6e4802c1adcd None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:14.985910 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b02d97f0-181e-488e-acec-6e4802c1adcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:15.076266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471297c-b95a-477a-a04c-c3ac91ecf7bf None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:15.077419 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:15.092743 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:15.092931 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 374/720] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:50:15.092931 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36ca87-e2b0-4f45-801d-30f416194cc5 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:15.093391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36ca87-e2b0-4f45-801d-30f416194cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:15.096711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36ca87-e2b0-4f45-801d-30f416194cc5 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:15.100539 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 375/721] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:15.105793 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4050cbd-8e86-43ef-b78d-862b8ca84763 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:15.106347 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4050cbd-8e86-43ef-b78d-862b8ca84763 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:15.106788 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4050cbd-8e86-43ef-b78d-862b8ca84763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:15.137145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d4050cbd-8e86-43ef-b78d-862b8ca84763 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:15.138579 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d4050cbd-8e86-43ef-b78d-862b8ca84763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:15.139900 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbd978-4716-40b5-9a5b-0cfb73c791c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:15.140812 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbd978-4716-40b5-9a5b-0cfb73c791c8 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:15.141521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbd978-4716-40b5-9a5b-0cfb73c791c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:15.141951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbd978-4716-40b5-9a5b-0cfb73c791c8 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:15.143617 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbd978-4716-40b5-9a5b-0cfb73c791c8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:15.148828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbd978-4716-40b5-9a5b-0cfb73c791c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:15.151559 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbd978-4716-40b5-9a5b-0cfb73c791c8 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:15.161059 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 347/722] 10.222.0.27 () {56 vars in 1124 bytes} [Sat Jul 16 14:50:14 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:15.166047 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbd978-4716-40b5-9a5b-0cfb73c791c8 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:15.302008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 376/723] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:50:15.328055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4050cbd-8e86-43ef-b78d-862b8ca84763 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:15.376050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbd978-4716-40b5-9a5b-0cfb73c791c8 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:15.431871 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e580ad-b1c0-4666-9e15-416f482aa22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:15.433068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e580ad-b1c0-4666-9e15-416f482aa22a None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:15.433357 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e580ad-b1c0-4666-9e15-416f482aa22a None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:15.433634 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e580ad-b1c0-4666-9e15-416f482aa22a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:15.451527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e580ad-b1c0-4666-9e15-416f482aa22a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:50:15.459538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-08e580ad-b1c0-4666-9e15-416f482aa22a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:50:15.459538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:15.459538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:15.459538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:15.459538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:15.459538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:15.459538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:15.459538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:15.459538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:15.459538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:15.459538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:15.459538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:15.459538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:50:15.459538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:50:15.459538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:50:15.459538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:50:15.461205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:50:15.461205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:50:15.461205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:50:15.461205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:50:15.461205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:15.461205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:15.461205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:15.461205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:15.461205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:15.461205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:15.461205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:15.461205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:15.461205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:15.461205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:15.461205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:15.461205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:15.462325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:15.462325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:15.462325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:15.462325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:15.462325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:50:15.462325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:50:15.462325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:50:15.462325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:50:15.462325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:50:15.462325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:50:15.462325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:50:15.462325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:15.462325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 348/724] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:50:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:15.471065 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e580ad-b1c0-4666-9e15-416f482aa22a None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:15.500222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4050cbd-8e86-43ef-b78d-862b8ca84763 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:15.540736 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:15.555924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:15.574120 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b68992-80b5-41f4-acdc-ff29b0742219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:15.574948 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b68992-80b5-41f4-acdc-ff29b0742219 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:15.575230 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b68992-80b5-41f4-acdc-ff29b0742219 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:15.575500 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b68992-80b5-41f4-acdc-ff29b0742219 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:15.593330 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b68992-80b5-41f4-acdc-ff29b0742219 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:50:15.601077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-21b68992-80b5-41f4-acdc-ff29b0742219 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:50:15.601077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:15.601077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:15.601077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:15.601077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:15.601077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:15.601077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:15.601077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:15.601077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:15.601077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:15.601077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:15.601077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:15.601077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:50:15.601077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:50:15.601077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:50:15.601077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:50:15.602504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:50:15.602504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:50:15.602504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:50:15.602504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:50:15.602504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:15.602504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:15.602504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:15.602504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:15.602504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:15.602504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:15.602504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:15.602504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:15.602504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:15.602504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:15.602504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:15.602504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:15.603504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:15.603504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:15.603504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:15.603504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:15.603504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:50:15.603504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:50:15.603504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:50:15.603504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:50:15.603504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:50:15.603504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:50:15.603504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:50:15.603504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:15.603504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 377/725] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:50:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:15.615935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b68992-80b5-41f4-acdc-ff29b0742219 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:15.657178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e580ad-b1c0-4666-9e15-416f482aa22a None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:15.697519 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54555c3c-a08d-47ce-972a-6ae03119e13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:15.698564 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54555c3c-a08d-47ce-972a-6ae03119e13d None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:15.699082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54555c3c-a08d-47ce-972a-6ae03119e13d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:15.699592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54555c3c-a08d-47ce-972a-6ae03119e13d None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:15.700554 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54555c3c-a08d-47ce-972a-6ae03119e13d None admin] RBAC: Adding query filter params (name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:15.701063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54555c3c-a08d-47ce-972a-6ae03119e13d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:15.702665 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54555c3c-a08d-47ce-972a-6ae03119e13d None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:15.713632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 349/726] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:50:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:15.720104 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54555c3c-a08d-47ce-972a-6ae03119e13d None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:15.859063 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b68992-80b5-41f4-acdc-ff29b0742219 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:15.931145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e66200-bb2d-43d4-9e1d-51a25982d6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:15.931768 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e66200-bb2d-43d4-9e1d-51a25982d6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:15.932051 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e66200-bb2d-43d4-9e1d-51a25982d6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:15.932329 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e66200-bb2d-43d4-9e1d-51a25982d6f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:15.933099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e66200-bb2d-43d4-9e1d-51a25982d6f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:15.933407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e66200-bb2d-43d4-9e1d-51a25982d6f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:15.934833 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e66200-bb2d-43d4-9e1d-51a25982d6f4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:15.939160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:15.956574 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54555c3c-a08d-47ce-972a-6ae03119e13d None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:15.959531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 378/727] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:50:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:15.963455 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e66200-bb2d-43d4-9e1d-51a25982d6f4 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:16.027155 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275eed05-0b7e-4db9-a88f-d53713e48cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:16.028355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275eed05-0b7e-4db9-a88f-d53713e48cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:16.028886 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275eed05-0b7e-4db9-a88f-d53713e48cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:16.029362 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275eed05-0b7e-4db9-a88f-d53713e48cd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:16.037675 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:16.043286 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275eed05-0b7e-4db9-a88f-d53713e48cd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:16.045074 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275eed05-0b7e-4db9-a88f-d53713e48cd4 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:16.065342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-275eed05-0b7e-4db9-a88f-d53713e48cd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:16.065342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:16.065342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:16.065342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:16.065342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:16.065342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:16.065342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:16.065342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:16.065342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:16.065342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:16.065342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:16.065342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:16.065342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:16.065342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:16.065342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:16.066349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:16.066349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:16.066349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:16.066349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:16.066349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:16.066349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:16.066349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:16.066349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:16.066349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:16.066349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:16.066349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:16.066349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:16.066349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:16.066349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:16.066349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:16.066349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:16.066349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:16.067320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:16.067320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:16.067320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:16.067320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:16.069648 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 350/728] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:16.074959 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275eed05-0b7e-4db9-a88f-d53713e48cd4 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:16.159594 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e66200-bb2d-43d4-9e1d-51a25982d6f4 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:16.192396 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275eed05-0b7e-4db9-a88f-d53713e48cd4 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:16.220968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff281f8-57af-43ff-a9f0-b84d1ab6e45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:16.221787 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff281f8-57af-43ff-a9f0-b84d1ab6e45f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:16.222071 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff281f8-57af-43ff-a9f0-b84d1ab6e45f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:16.222347 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff281f8-57af-43ff-a9f0-b84d1ab6e45f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:16.234289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90b7ad-d97b-41e8-af4e-373fcae93db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:16.235471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90b7ad-d97b-41e8-af4e-373fcae93db7 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:16.235958 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90b7ad-d97b-41e8-af4e-373fcae93db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:16.236433 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d90b7ad-d97b-41e8-af4e-373fcae93db7 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:16.237405 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90b7ad-d97b-41e8-af4e-373fcae93db7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:16.237915 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90b7ad-d97b-41e8-af4e-373fcae93db7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:16.239584 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d90b7ad-d97b-41e8-af4e-373fcae93db7 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:16.253624 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 351/729] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:16.262311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90b7ad-d97b-41e8-af4e-373fcae93db7 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:16.262523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff281f8-57af-43ff-a9f0-b84d1ab6e45f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:16.265257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff281f8-57af-43ff-a9f0-b84d1ab6e45f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:16.283345 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:16.314097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-0ff281f8-57af-43ff-a9f0-b84d1ab6e45f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:16.314097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:16.314097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:16.314097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:16.314097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:16.314097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:16.314097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:16.314097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:16.314097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:16.314097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:16.314097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:16.314097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:16.314097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:16.314097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:16.314097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:16.315201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:16.315201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:16.315201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:16.315201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:16.315201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:16.315201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:16.315201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:16.315201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:16.315201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:16.315201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:16.315201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:16.315201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:16.315201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:16.315201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:16.315201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:16.315201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:16.315201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:16.316131 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:16.316131 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:16.316131 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:16.316131 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:16.321489 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 379/730] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:16.330031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff281f8-57af-43ff-a9f0-b84d1ab6e45f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:16.512971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d90b7ad-d97b-41e8-af4e-373fcae93db7 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:16.563770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:16.571937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff281f8-57af-43ff-a9f0-b84d1ab6e45f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:16.605163 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832e42c-5c09-444a-934f-6c0d9d8b9e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:16.606548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832e42c-5c09-444a-934f-6c0d9d8b9e4d None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:16.607081 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832e42c-5c09-444a-934f-6c0d9d8b9e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:16.607565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832e42c-5c09-444a-934f-6c0d9d8b9e4d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:16.643834 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832e42c-5c09-444a-934f-6c0d9d8b9e4d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:16.646578 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832e42c-5c09-444a-934f-6c0d9d8b9e4d None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:16.687619 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deae58e-132d-4012-bf9d-a25536873b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:16.688741 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deae58e-132d-4012-bf9d-a25536873b11 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:16.689209 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deae58e-132d-4012-bf9d-a25536873b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:16.689699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deae58e-132d-4012-bf9d-a25536873b11 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:16.691374 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deae58e-132d-4012-bf9d-a25536873b11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:16.691904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deae58e-132d-4012-bf9d-a25536873b11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:16.694307 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deae58e-132d-4012-bf9d-a25536873b11 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:16.706833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-7832e42c-5c09-444a-934f-6c0d9d8b9e4d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 14:50:16.706833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:16.706833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:16.706833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:16.706833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:16.706833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:16.706833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:16.706833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:16.706833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:16.706833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:16.706833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:16.706833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:16.706833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:50:16.706833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:50:16.706833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:50:16.706833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:50:16.709159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:16.709159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:16.709159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:50:16.709159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:16.709159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:50:16.709159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:16.709159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:16.709159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:16.709159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:16.709159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:16.709159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:16.709159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:16.709159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:16.709159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:16.709159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:16.709159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:16.709159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:16.709159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:16.711630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:16.711630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:16.711630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:50:16.711630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:50:16.711630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:50:16.711630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:50:16.711630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:50:16.711630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:50:16.711630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 14:50:16.711630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:16.712998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 352/731] 10.222.0.27 () {56 vars in 1117 bytes} [Sat Jul 16 14:50:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 451 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:16.723431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832e42c-5c09-444a-934f-6c0d9d8b9e4d None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:16.747283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 380/732] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:16.754399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deae58e-132d-4012-bf9d-a25536873b11 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:16.915378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:16.985762 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deae58e-132d-4012-bf9d-a25536873b11 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:17.054847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832e42c-5c09-444a-934f-6c0d9d8b9e4d None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:17.103473 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38331d86-e788-46a6-a60d-f8daa6ba0b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:17.104663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38331d86-e788-46a6-a60d-f8daa6ba0b1c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:17.105080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38331d86-e788-46a6-a60d-f8daa6ba0b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:17.105359 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38331d86-e788-46a6-a60d-f8daa6ba0b1c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:17.127363 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38331d86-e788-46a6-a60d-f8daa6ba0b1c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:17.134923 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38331d86-e788-46a6-a60d-f8daa6ba0b1c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:17.155392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-38331d86-e788-46a6-a60d-f8daa6ba0b1c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 16 14:50:17.155392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:17.155392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:17.155392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:17.155392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:17.155392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:17.155392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:17.155392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:17.155392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:17.155392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:17.155392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:17.155392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:17.155392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:50:17.155392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:50:17.155392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:50:17.155392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:50:17.157291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:17.157291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:17.157291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:50:17.157291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:17.157291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:50:17.157291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:17.157291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:17.157291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:17.157291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:17.157291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:17.157291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:17.157291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:17.157291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:17.157291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:17.157291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:17.157291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:17.157291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:17.157291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:17.159132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:17.159132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:17.159132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:50:17.159132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:50:17.159132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:50:17.159132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:50:17.159132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:50:17.159132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:50:17.159132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 16 14:50:17.159132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:17.159132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 381/733] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:50:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:17.163895 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec670c5-641b-44c9-8b8c-58334b33aae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:17.164628 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec670c5-641b-44c9-8b8c-58334b33aae3 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:17.165034 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec670c5-641b-44c9-8b8c-58334b33aae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:17.165296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec670c5-641b-44c9-8b8c-58334b33aae3 None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:17.166173 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec670c5-641b-44c9-8b8c-58334b33aae3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:17.166483 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec670c5-641b-44c9-8b8c-58334b33aae3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:17.167775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38331d86-e788-46a6-a60d-f8daa6ba0b1c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:17.168042 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec670c5-641b-44c9-8b8c-58334b33aae3 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:17.207468 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:17.218267 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 353/734] 10.222.0.27 () {56 vars in 1149 bytes} [Sat Jul 16 14:50:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 496 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:17.220150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bd532-6dc7-47c4-af6d-7476f854066d None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:17.220491 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bd532-6dc7-47c4-af6d-7476f854066d None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:17.220773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bd532-6dc7-47c4-af6d-7476f854066d None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:17.223041 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 354/735] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:17.228909 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bd532-6dc7-47c4-af6d-7476f854066d None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:17.346632 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38331d86-e788-46a6-a60d-f8daa6ba0b1c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:17.378003 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bd532-6dc7-47c4-af6d-7476f854066d None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:17.410101 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:17.424800 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d6105-9f65-4ee3-b09a-a7ebcb25efcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:17.425138 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d7e8c-7083-417b-904e-24351294780b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:17.425465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d6105-9f65-4ee3-b09a-a7ebcb25efcb None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:17.425746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d6105-9f65-4ee3-b09a-a7ebcb25efcb None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:17.425916 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d7e8c-7083-417b-904e-24351294780b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:17.426012 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d6105-9f65-4ee3-b09a-a7ebcb25efcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:17.426261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d7e8c-7083-417b-904e-24351294780b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:17.426467 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d7e8c-7083-417b-904e-24351294780b None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:17.427672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d7e8c-7083-417b-904e-24351294780b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:17.428180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d7e8c-7083-417b-904e-24351294780b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:17.431205 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d7e8c-7083-417b-904e-24351294780b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:17.457732 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 382/736] 10.222.0.27 () {56 vars in 1135 bytes} [Sat Jul 16 14:50:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:17.459171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2320bbd-d6a5-4c5f-9c7a-0015564a08cc None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:17.459459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2320bbd-d6a5-4c5f-9c7a-0015564a08cc None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:17.459733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2320bbd-d6a5-4c5f-9c7a-0015564a08cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:17.480394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a2320bbd-d6a5-4c5f-9c7a-0015564a08cc None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:17.481618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a2320bbd-d6a5-4c5f-9c7a-0015564a08cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:17.531062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d6105-9f65-4ee3-b09a-a7ebcb25efcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:17.533028 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d6105-9f65-4ee3-b09a-a7ebcb25efcb None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:17.562582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 383/737] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:50:17.563743 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2320bbd-d6a5-4c5f-9c7a-0015564a08cc None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:17.575808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:17.681854 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 355/738] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:50:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:50:17.688097 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d6105-9f65-4ee3-b09a-a7ebcb25efcb None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:17.704775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2320bbd-d6a5-4c5f-9c7a-0015564a08cc None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:17.723959 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:17.753674 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a210bab-f96a-497a-9a56-70b6f33cd381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:17.755206 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a210bab-f96a-497a-9a56-70b6f33cd381 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:17.755735 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a210bab-f96a-497a-9a56-70b6f33cd381 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:17.756014 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a210bab-f96a-497a-9a56-70b6f33cd381 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:17.845238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a210bab-f96a-497a-9a56-70b6f33cd381 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:17.846805 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a210bab-f96a-497a-9a56-70b6f33cd381 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:17.859272 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:17.862702 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d6105-9f65-4ee3-b09a-a7ebcb25efcb None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:17.920632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea37498-5d1a-4dae-b1d3-509726faede4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:17.921442 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea37498-5d1a-4dae-b1d3-509726faede4 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:17.921793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea37498-5d1a-4dae-b1d3-509726faede4 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:17.922137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea37498-5d1a-4dae-b1d3-509726faede4 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:17.922963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea37498-5d1a-4dae-b1d3-509726faede4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:17.924343 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea37498-5d1a-4dae-b1d3-509726faede4 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:17.973278 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 384/739] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:50:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 16 14:50:17.974048 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a210bab-f96a-497a-9a56-70b6f33cd381 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:18.021754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 356/740] 10.222.0.27 () {60 vars in 1141 bytes} [Sat Jul 16 14:50:17 2022] POST /identity/v3/services => generated 240 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:50:18.024564 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832880e0-3f83-43dd-a7eb-6fd908b8c461 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:18.024856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832880e0-3f83-43dd-a7eb-6fd908b8c461 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:18.025133 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832880e0-3f83-43dd-a7eb-6fd908b8c461 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:18.026665 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 357/741] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:18.028058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b81d7-b112-44f0-a52f-aaa124e09ad2 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:18.028347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b81d7-b112-44f0-a52f-aaa124e09ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:18.028622 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b81d7-b112-44f0-a52f-aaa124e09ad2 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:18.030391 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 358/742] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:18.032458 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b81d7-b112-44f0-a52f-aaa124e09ad2 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:18.133991 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a210bab-f96a-497a-9a56-70b6f33cd381 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:18.144456 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:18.154412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:18.201813 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa97da4-49cb-4ec4-b9a5-7a6c938a545c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:18.202048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b81d7-b112-44f0-a52f-aaa124e09ad2 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:18.203208 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa97da4-49cb-4ec4-b9a5-7a6c938a545c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:18.203721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa97da4-49cb-4ec4-b9a5-7a6c938a545c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:18.204194 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa97da4-49cb-4ec4-b9a5-7a6c938a545c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:18.221424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa97da4-49cb-4ec4-b9a5-7a6c938a545c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:18.223736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa97da4-49cb-4ec4-b9a5-7a6c938a545c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:18.256462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-4aa97da4-49cb-4ec4-b9a5-7a6c938a545c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:18.256462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:18.256462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:18.256462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:18.256462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:18.256462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:18.256462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:18.256462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:18.256462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:18.256462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:18.256462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:18.256462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:18.256462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:18.256462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:18.256462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:18.258227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:18.258227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:18.258227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:18.258227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:18.258227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:18.258227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:18.258227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:18.258227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:18.258227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:18.258227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:18.258227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:18.258227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:18.258227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:18.258227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:18.258227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:18.258227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:18.258227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:18.259542 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:18.259542 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:18.259542 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:18.259542 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:18.259542 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 385/743] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:18.260096 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f0bf7-1a0f-4a04-b740-86b4f0a96b94 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:18.260577 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f0bf7-1a0f-4a04-b740-86b4f0a96b94 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:18.261048 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f0bf7-1a0f-4a04-b740-86b4f0a96b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:18.268973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0619caa-34a5-41cc-a353-44c92d936009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:18.269836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0619caa-34a5-41cc-a353-44c92d936009 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:18.270125 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0619caa-34a5-41cc-a353-44c92d936009 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:18.270401 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0619caa-34a5-41cc-a353-44c92d936009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:18.283027 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0619caa-34a5-41cc-a353-44c92d936009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:18.283316 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ad1f0bf7-1a0f-4a04-b740-86b4f0a96b94 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:18.284536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0619caa-34a5-41cc-a353-44c92d936009 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:18.284701 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ad1f0bf7-1a0f-4a04-b740-86b4f0a96b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:18.305419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-f0619caa-34a5-41cc-a353-44c92d936009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:18.305419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:18.305419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:18.305419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:18.305419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:18.305419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:18.305419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:18.305419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:18.305419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:18.305419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:18.305419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:18.305419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:18.305419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:18.305419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:18.305419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:18.306496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:18.306496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:18.306496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:18.306496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:18.306496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:18.306496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:18.306496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:18.306496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:18.306496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:18.306496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:18.306496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:18.306496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:18.306496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:18.306496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:18.306496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:18.306496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:18.306496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:18.307934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:18.307934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:18.307934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:18.307934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:18.307934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 359/744] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:18.308440 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f559c08-66fa-4257-b0f7-76ea6a9e2970 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:18.311003 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f559c08-66fa-4257-b0f7-76ea6a9e2970 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:18.311003 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f559c08-66fa-4257-b0f7-76ea6a9e2970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:18.328454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-0f559c08-66fa-4257-b0f7-76ea6a9e2970 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:18.329922 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-0f559c08-66fa-4257-b0f7-76ea6a9e2970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:18.383775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 386/745] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:18 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:50:18.384965 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f0bf7-1a0f-4a04-b740-86b4f0a96b94 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:18.414382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 360/746] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:18 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 16 14:50:18.415606 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f559c08-66fa-4257-b0f7-76ea6a9e2970 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:18.496418 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:18.537641 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:18.550652 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f559c08-66fa-4257-b0f7-76ea6a9e2970 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:18.562858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f0bf7-1a0f-4a04-b740-86b4f0a96b94 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:18.603669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd28ea-c965-41e5-8cfa-430016499c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:18.604783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd28ea-c965-41e5-8cfa-430016499c89 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:18.605240 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efab161-2178-4fad-9211-b21c44769a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:18.605684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd28ea-c965-41e5-8cfa-430016499c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:18.605761 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd28ea-c965-41e5-8cfa-430016499c89 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:18.605945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efab161-2178-4fad-9211-b21c44769a9e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:18.606189 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efab161-2178-4fad-9211-b21c44769a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:18.606465 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efab161-2178-4fad-9211-b21c44769a9e None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:18.606953 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd28ea-c965-41e5-8cfa-430016499c89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:18.607240 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efab161-2178-4fad-9211-b21c44769a9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:18.607333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd28ea-c965-41e5-8cfa-430016499c89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:18.607549 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efab161-2178-4fad-9211-b21c44769a9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:18.608888 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd28ea-c965-41e5-8cfa-430016499c89 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:18.608986 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efab161-2178-4fad-9211-b21c44769a9e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:18.621980 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 361/747] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:18.626877 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 387/748] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:18.626877 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efab161-2178-4fad-9211-b21c44769a9e None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:18.627876 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd28ea-c965-41e5-8cfa-430016499c89 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:18.767314 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efab161-2178-4fad-9211-b21c44769a9e None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:18.780942 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd28ea-c965-41e5-8cfa-430016499c89 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:18.805332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:18.821215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39ddde-4180-403c-a3ba-67f132060b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:18.822019 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39ddde-4180-403c-a3ba-67f132060b35 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:18.822305 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39ddde-4180-403c-a3ba-67f132060b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:18.822576 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39ddde-4180-403c-a3ba-67f132060b35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:18.823190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff70f7c-ad2d-46a4-b368-86b2a9d53b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:18.824450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff70f7c-ad2d-46a4-b368-86b2a9d53b84 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:18.824801 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff70f7c-ad2d-46a4-b368-86b2a9d53b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:18.825246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff70f7c-ad2d-46a4-b368-86b2a9d53b84 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:18.826615 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff70f7c-ad2d-46a4-b368-86b2a9d53b84 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:18.829274 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff70f7c-ad2d-46a4-b368-86b2a9d53b84 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:18.836056 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39ddde-4180-403c-a3ba-67f132060b35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:50:18.844161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-bc39ddde-4180-403c-a3ba-67f132060b35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:50:18.844161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:18.844161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:18.844161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:18.844161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:18.844161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:18.844161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:18.844161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:18.844161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:18.844161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:18.844161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:18.844161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:18.844161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:50:18.844161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:50:18.844161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:50:18.844161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:50:18.845884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:50:18.845884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:50:18.845884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:50:18.845884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:50:18.845884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:18.845884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:18.845884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:18.845884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:18.845884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:18.845884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:18.845884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:18.845884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:18.845884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:18.845884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:18.845884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:18.845884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:18.847116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:18.847116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:18.847116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:18.847116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:18.847116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:50:18.847116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:50:18.847116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:50:18.847116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:50:18.847116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:50:18.847116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:50:18.847116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:50:18.847116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:18.847116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 362/749] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:50:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:18.847116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39ddde-4180-403c-a3ba-67f132060b35 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:18.969852 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39ddde-4180-403c-a3ba-67f132060b35 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:18.971486 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 388/750] 10.222.0.27 () {60 vars in 1141 bytes} [Sat Jul 16 14:50:18 2022] POST /identity/v3/services => generated 251 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:50:18.973470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff70f7c-ad2d-46a4-b368-86b2a9d53b84 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:19.008950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:19.031197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cdf58-df4a-46f3-a3ad-b6ca36f5b5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:19.031591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cdf58-df4a-46f3-a3ad-b6ca36f5b5f5 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:19.033422 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cdf58-df4a-46f3-a3ad-b6ca36f5b5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:19.033422 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cdf58-df4a-46f3-a3ad-b6ca36f5b5f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:19.048153 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98cdf58-df4a-46f3-a3ad-b6ca36f5b5f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:19.051120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98cdf58-df4a-46f3-a3ad-b6ca36f5b5f5 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:19.076943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-f98cdf58-df4a-46f3-a3ad-b6ca36f5b5f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:19.076943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:19.076943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:19.076943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:19.076943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:19.076943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:19.076943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:19.076943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:19.076943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:19.076943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:19.076943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:19.076943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:19.076943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:50:19.076943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:50:19.076943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:50:19.077916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:50:19.077916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:19.077916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:19.077916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:19.077916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:19.077916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:19.077916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:19.077916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:19.077916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:19.077916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:19.077916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:19.077916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:19.077916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:19.077916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:19.077916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:19.077916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:19.077916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:19.078828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:50:19.078828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:50:19.078828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:50:19.078828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:50:19.078828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:50:19.078828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:50:19.078828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:19.078828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:19.083831 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 363/751] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:19.085966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cdf58-df4a-46f3-a3ad-b6ca36f5b5f5 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:19.166106 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:19.203338 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff70f7c-ad2d-46a4-b368-86b2a9d53b84 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:19.275716 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30728c-c849-4962-a90e-1e70ccc7d04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:19.277250 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30728c-c849-4962-a90e-1e70ccc7d04e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:19.277670 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30728c-c849-4962-a90e-1e70ccc7d04e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:19.278079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30728c-c849-4962-a90e-1e70ccc7d04e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:19.296552 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30728c-c849-4962-a90e-1e70ccc7d04e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:19.299794 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30728c-c849-4962-a90e-1e70ccc7d04e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:19.310197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cdf58-df4a-46f3-a3ad-b6ca36f5b5f5 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:19.321171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-cd30728c-c849-4962-a90e-1e70ccc7d04e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:19.321171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:19.321171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:19.321171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:19.321171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:19.321171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:19.321171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:19.321171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:19.321171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:19.321171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:19.321171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:19.321171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:19.321171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:50:19.321171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:50:19.321171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:50:19.322117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:50:19.322117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:19.322117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:19.322117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:19.322117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:19.322117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:19.322117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:19.322117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:19.322117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:19.322117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:19.322117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:19.322117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:19.322117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:19.322117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:19.322117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:19.322117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:19.322117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:19.323025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:50:19.323025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:50:19.323025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:50:19.323025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:50:19.323025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:50:19.323025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:50:19.323025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:19.323025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:19.326509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 389/752] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:19.327717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30728c-c849-4962-a90e-1e70ccc7d04e None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:19.391144 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69d05f-5930-4c0c-a732-3f962a69f3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:19.392285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69d05f-5930-4c0c-a732-3f962a69f3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:19.392800 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69d05f-5930-4c0c-a732-3f962a69f3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:19.393253 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69d05f-5930-4c0c-a732-3f962a69f3f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:19.395338 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69d05f-5930-4c0c-a732-3f962a69f3f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:19.395850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69d05f-5930-4c0c-a732-3f962a69f3f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:19.400243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69d05f-5930-4c0c-a732-3f962a69f3f0 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:19.418318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 364/753] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:50:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:19.419733 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69d05f-5930-4c0c-a732-3f962a69f3f0 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:19.449297 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:19.522295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30728c-c849-4962-a90e-1e70ccc7d04e None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:19.597142 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1b781-e461-4c94-a7ba-6dec525c718f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:19.598768 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1b781-e461-4c94-a7ba-6dec525c718f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:19.599619 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1b781-e461-4c94-a7ba-6dec525c718f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:19.600407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1b781-e461-4c94-a7ba-6dec525c718f None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:19.603913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1b781-e461-4c94-a7ba-6dec525c718f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:19.604776 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1b781-e461-4c94-a7ba-6dec525c718f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:19.607963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1b781-e461-4c94-a7ba-6dec525c718f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:19.614397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69d05f-5930-4c0c-a732-3f962a69f3f0 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:19.634514 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 390/754] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:50:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:19.636354 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1b781-e461-4c94-a7ba-6dec525c718f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:19.651322 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:19.691341 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e5c24-7335-4d0e-b089-ec13c8dae479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:19.692741 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e5c24-7335-4d0e-b089-ec13c8dae479 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:19.693750 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e5c24-7335-4d0e-b089-ec13c8dae479 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:19.695786 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e5c24-7335-4d0e-b089-ec13c8dae479 None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:19.698682 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e5c24-7335-4d0e-b089-ec13c8dae479 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:19.699255 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e5c24-7335-4d0e-b089-ec13c8dae479 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:19.703006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e5c24-7335-4d0e-b089-ec13c8dae479 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:19.738916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 365/755] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:50:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:19.738916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e5c24-7335-4d0e-b089-ec13c8dae479 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:19.888671 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1b781-e461-4c94-a7ba-6dec525c718f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:19.953206 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:19.976663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b4323-fc8f-4195-8a13-66ab54282c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:19.978099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b4323-fc8f-4195-8a13-66ab54282c24 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:19.978970 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b4323-fc8f-4195-8a13-66ab54282c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:19.979457 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b4323-fc8f-4195-8a13-66ab54282c24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:19.982863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e5c24-7335-4d0e-b089-ec13c8dae479 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:20.006374 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b4323-fc8f-4195-8a13-66ab54282c24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:20.010364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b4323-fc8f-4195-8a13-66ab54282c24 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:20.047831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-6f7b4323-fc8f-4195-8a13-66ab54282c24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:20.047831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:20.047831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:20.047831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:20.047831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:20.047831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:20.047831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:20.047831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:20.047831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:20.047831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:20.047831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:20.047831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:20.047831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:20.047831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:20.047831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:20.049604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:20.049604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:20.049604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:20.049604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:20.049604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:20.049604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:20.049604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:20.049604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:20.049604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:20.049604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:20.049604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:20.049604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:20.049604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:20.049604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:20.049604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:20.049604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:20.049604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:20.051289 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:20.051289 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:20.051289 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:20.051289 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:20.055515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 391/756] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:20.057709 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b4323-fc8f-4195-8a13-66ab54282c24 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:20.059929 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76759b77-2c39-4589-8c76-741b513b4f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:20.061058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76759b77-2c39-4589-8c76-741b513b4f19 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:20.061544 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76759b77-2c39-4589-8c76-741b513b4f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:20.062026 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76759b77-2c39-4589-8c76-741b513b4f19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:20.063681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76759b77-2c39-4589-8c76-741b513b4f19 None admin] RBAC: Adding query filter params (role.id=0efeff46573247f3b9f1e2dec2968672, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=e1464c83fb294a3d90e5763ef648e834) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:20.064692 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76759b77-2c39-4589-8c76-741b513b4f19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:20.068603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76759b77-2c39-4589-8c76-741b513b4f19 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:20.087009 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 366/757] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:50:19 2022] GET /identity/v3/role_assignments?user.id=e1464c83fb294a3d90e5763ef648e834&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=0efeff46573247f3b9f1e2dec2968672 => generated 259 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:20.087009 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76759b77-2c39-4589-8c76-741b513b4f19 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:20.177429 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:20.315191 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b4323-fc8f-4195-8a13-66ab54282c24 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:20.401050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8154c4-b664-4122-add1-26f412e0ee89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:20.401735 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8154c4-b664-4122-add1-26f412e0ee89 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:20.402014 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8154c4-b664-4122-add1-26f412e0ee89 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:20.402280 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8154c4-b664-4122-add1-26f412e0ee89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:20.403180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8154c4-b664-4122-add1-26f412e0ee89 None admin] RBAC: Adding query filter params (role.id=21119ebd056740048b9ec639ebb20446, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=a4f499707998403d8f833eb783599466) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:20.403494 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8154c4-b664-4122-add1-26f412e0ee89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:20.404976 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8154c4-b664-4122-add1-26f412e0ee89 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:20.412530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 392/758] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:50:20 2022] GET /identity/v3/role_assignments?user.id=a4f499707998403d8f833eb783599466&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=21119ebd056740048b9ec639ebb20446 => generated 259 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:20.427137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76759b77-2c39-4589-8c76-741b513b4f19 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:20.502363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a605b-9f7b-413a-9888-b665c98e8f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:20.503518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a605b-9f7b-413a-9888-b665c98e8f61 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:20.504033 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a605b-9f7b-413a-9888-b665c98e8f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:20.504525 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a605b-9f7b-413a-9888-b665c98e8f61 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:20.506000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a605b-9f7b-413a-9888-b665c98e8f61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:20.506577 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a605b-9f7b-413a-9888-b665c98e8f61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:20.509971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a605b-9f7b-413a-9888-b665c98e8f61 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:20.533866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 367/759] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:20.542012 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8154c4-b664-4122-add1-26f412e0ee89 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:20.549163 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354745e-f888-4fff-b413-313099933e35 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:20.551802 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354745e-f888-4fff-b413-313099933e35 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:20.552749 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354745e-f888-4fff-b413-313099933e35 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:20.556164 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 368/760] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:20.566450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd417b-b667-4e9f-b659-4bd84517144c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:20.567492 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd417b-b667-4e9f-b659-4bd84517144c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:20.568301 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd417b-b667-4e9f-b659-4bd84517144c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:20.609610 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-56fd417b-b667-4e9f-b659-4bd84517144c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:20.610932 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-56fd417b-b667-4e9f-b659-4bd84517144c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:20.656925 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:20.726607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 369/761] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:20.751689 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd417b-b667-4e9f-b659-4bd84517144c None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:20.770073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:20.787791 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8154c4-b664-4122-add1-26f412e0ee89 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:20.875084 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd417b-b667-4e9f-b659-4bd84517144c None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:20.917865 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861ebac-8b82-4edf-a9ab-c7fb1e13acdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:20.918699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861ebac-8b82-4edf-a9ab-c7fb1e13acdc None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:20.919010 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861ebac-8b82-4edf-a9ab-c7fb1e13acdc None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:20.919274 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861ebac-8b82-4edf-a9ab-c7fb1e13acdc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:20.931202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6861ebac-8b82-4edf-a9ab-c7fb1e13acdc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:20.932749 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6861ebac-8b82-4edf-a9ab-c7fb1e13acdc None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:20.954129 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68963d1-cd40-4454-927a-164b805eeba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:20.954972 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68963d1-cd40-4454-927a-164b805eeba9 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:20.955254 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68963d1-cd40-4454-927a-164b805eeba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:20.955529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68963d1-cd40-4454-927a-164b805eeba9 None admin] PATH_INFO: `/v3/services/network` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:20.956207 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68963d1-cd40-4454-927a-164b805eeba9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:20.957661 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68963d1-cd40-4454-927a-164b805eeba9 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:20.970406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-6861ebac-8b82-4edf-a9ab-c7fb1e13acdc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 16 14:50:20.970406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:20.970406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:20.970406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:20.970406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:20.970406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:20.970406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:20.970406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:20.970406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:20.970406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:20.970406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:20.970406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:20.970406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:50:20.970406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:50:20.970406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:50:20.970406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:50:20.972239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:20.972239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:20.972239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:50:20.972239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:20.972239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:50:20.972239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:20.972239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:20.972239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:20.972239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:20.972239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:20.972239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:20.972239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:20.972239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:20.972239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:20.972239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:20.972239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:20.972239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:20.972239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:20.974292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:20.974292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:20.974292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:50:20.974292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:50:20.974292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:50:20.974292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:50:20.974292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:50:20.974292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:50:20.974292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 16 14:50:20.974292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:20.977068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 393/762] 10.222.0.27 () {56 vars in 1093 bytes} [Sat Jul 16 14:50:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:20.982791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-c68963d1-cd40-4454-927a-164b805eeba9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 16 14:50:20.982791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:20.982791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:20.982791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:20.982791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:20.982791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:20.982791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:20.982791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:20.982791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 16 14:50:20.982791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:20.982791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:20.982791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:20.982791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:20.982791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:20.982791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:20.982791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:20.982791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:20.987459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:20.987459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:20.987459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:20.987459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:20.987459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:20.987459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:20.987459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:20.987459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:20.987459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:20.987459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:20.987459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:20.987459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:20.987459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:20.987459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:20.987459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:20.987459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:20.992829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:20.992829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:20.992829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:20.992829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:20.992829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:20.992829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:20.992829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:20.992829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:20.992829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:20.992829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:20.992829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:20.992829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:20.992829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 16 14:50:20.992829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:21.001306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861ebac-8b82-4edf-a9ab-c7fb1e13acdc None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:21.001545 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 370/763] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:20 2022] GET /identity/v3/services/network => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:21.006955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68963d1-cd40-4454-927a-164b805eeba9 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:21.126310 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:21.154716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68963d1-cd40-4454-927a-164b805eeba9 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:21.197631 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502f6bf-555c-48d7-bcb8-5c6fc64740b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:21.198366 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502f6bf-555c-48d7-bcb8-5c6fc64740b4 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:21.200208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502f6bf-555c-48d7-bcb8-5c6fc64740b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:21.200208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502f6bf-555c-48d7-bcb8-5c6fc64740b4 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:21.200208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502f6bf-555c-48d7-bcb8-5c6fc64740b4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:21.200208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502f6bf-555c-48d7-bcb8-5c6fc64740b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:21.201401 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502f6bf-555c-48d7-bcb8-5c6fc64740b4 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:21.207425 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 371/764] 10.222.0.27 () {56 vars in 1112 bytes} [Sat Jul 16 14:50:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:21.214188 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502f6bf-555c-48d7-bcb8-5c6fc64740b4 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:21.311175 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861ebac-8b82-4edf-a9ab-c7fb1e13acdc None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:21.314301 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:21.332000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502f6bf-555c-48d7-bcb8-5c6fc64740b4 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:21.371274 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040366b7-eb3c-4907-b9c9-646d2af45d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:21.371957 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040366b7-eb3c-4907-b9c9-646d2af45d2d None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:21.372244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040366b7-eb3c-4907-b9c9-646d2af45d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:21.372517 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040366b7-eb3c-4907-b9c9-646d2af45d2d None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:21.373238 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040366b7-eb3c-4907-b9c9-646d2af45d2d None admin] RBAC: Adding query filter params (type=network) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:21.373539 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040366b7-eb3c-4907-b9c9-646d2af45d2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:21.375053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040366b7-eb3c-4907-b9c9-646d2af45d2d None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:21.381312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 372/765] 10.222.0.27 () {56 vars in 1112 bytes} [Sat Jul 16 14:50:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:21.387810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040366b7-eb3c-4907-b9c9-646d2af45d2d None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:21.392629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:21.430409 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedf34f-ada8-449d-8653-0bb02e06dc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:21.431137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedf34f-ada8-449d-8653-0bb02e06dc12 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:21.432312 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedf34f-ada8-449d-8653-0bb02e06dc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:21.432601 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedf34f-ada8-449d-8653-0bb02e06dc12 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:21.433476 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bedf34f-ada8-449d-8653-0bb02e06dc12 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:21.433783 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bedf34f-ada8-449d-8653-0bb02e06dc12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:21.435277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bedf34f-ada8-449d-8653-0bb02e06dc12 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:21.477025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 394/766] 10.222.0.27 () {56 vars in 1133 bytes} [Sat Jul 16 14:50:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 494 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:21.483546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedf34f-ada8-449d-8653-0bb02e06dc12 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:21.620581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040366b7-eb3c-4907-b9c9-646d2af45d2d None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:21.707024 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf9917-b823-407a-bacb-e2cfd1001ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:21.707024 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf9917-b823-407a-bacb-e2cfd1001ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:21.707024 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf9917-b823-407a-bacb-e2cfd1001ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:21.707024 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf9917-b823-407a-bacb-e2cfd1001ee5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:21.707632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf9917-b823-407a-bacb-e2cfd1001ee5 None admin] RBAC: Adding query filter params (interface=public, service_id=67097e6127b44b2abe929fe458e2f77f, region_id=RegionOne) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:21.707632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf9917-b823-407a-bacb-e2cfd1001ee5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:21.707632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf9917-b823-407a-bacb-e2cfd1001ee5 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:21.723950 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 373/767] 10.222.0.27 () {56 vars in 1251 bytes} [Sat Jul 16 14:50:21 2022] GET /identity/v3/endpoints?service_id=67097e6127b44b2abe929fe458e2f77f&interface=public®ion_id=RegionOne => generated 194 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:21.730305 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf9917-b823-407a-bacb-e2cfd1001ee5 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:21.799957 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedf34f-ada8-449d-8653-0bb02e06dc12 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:21.872937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0048450-7d37-4591-ac43-572d8e077d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:21.874614 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0048450-7d37-4591-ac43-572d8e077d32 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:21.875392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0048450-7d37-4591-ac43-572d8e077d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:21.876142 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0048450-7d37-4591-ac43-572d8e077d32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:21.910044 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0048450-7d37-4591-ac43-572d8e077d32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:21.913626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0048450-7d37-4591-ac43-572d8e077d32 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:21.927976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:21.955187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-e0048450-7d37-4591-ac43-572d8e077d32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:21.955187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:21.955187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:21.955187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:21.955187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:21.955187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:21.955187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:21.955187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:21.955187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:21.955187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:21.955187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:21.955187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:21.955187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:21.955187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:21.955187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:21.957277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:21.957277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:21.957277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:21.957277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:21.957277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:21.957277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:21.957277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:21.957277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:21.957277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:21.957277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:21.957277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:21.957277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:21.957277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:21.957277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:21.957277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:21.957277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:21.957277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:21.959340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:21.959340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:21.959340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:21.959340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:21.964128 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 395/768] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 481 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:21.971008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0048450-7d37-4591-ac43-572d8e077d32 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:21.999937 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf9917-b823-407a-bacb-e2cfd1001ee5 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:22.011082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:22.080430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd385a53-f46e-4283-98ac-3a54e614a5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:22.081593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd385a53-f46e-4283-98ac-3a54e614a5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:22.082087 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd385a53-f46e-4283-98ac-3a54e614a5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:22.082573 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd385a53-f46e-4283-98ac-3a54e614a5a4 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:22.083911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd385a53-f46e-4283-98ac-3a54e614a5a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:22.087538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd385a53-f46e-4283-98ac-3a54e614a5a4 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:22.104555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 374/769] 10.222.0.27 () {56 vars in 1087 bytes} [Sat Jul 16 14:50:21 2022] GET /identity/v3/services => generated 772 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:22.228485 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162e8f3-00ff-476e-8cac-c1daa054e658 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:22.228864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162e8f3-00ff-476e-8cac-c1daa054e658 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:22.229318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162e8f3-00ff-476e-8cac-c1daa054e658 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:22.232448 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0048450-7d37-4591-ac43-572d8e077d32 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:22.238619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 375/770] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:22.241681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81793e97-1834-4ad4-955f-0390d0344202 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:22.242179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81793e97-1834-4ad4-955f-0390d0344202 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:22.242666 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81793e97-1834-4ad4-955f-0390d0344202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:22.276112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-81793e97-1834-4ad4-955f-0390d0344202 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:22.278264 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-81793e97-1834-4ad4-955f-0390d0344202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:22.314779 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b65493-09a3-43f0-96c4-6850033f44d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:22.316317 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b65493-09a3-43f0-96c4-6850033f44d6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:22.317056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b65493-09a3-43f0-96c4-6850033f44d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:22.317780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b65493-09a3-43f0-96c4-6850033f44d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:22.320064 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b65493-09a3-43f0-96c4-6850033f44d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:22.320852 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b65493-09a3-43f0-96c4-6850033f44d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:22.323549 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b65493-09a3-43f0-96c4-6850033f44d6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:22.346150 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 396/771] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:22.352730 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b65493-09a3-43f0-96c4-6850033f44d6 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:22.536329 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:22.587139 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 376/772] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:50:22.600180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b65493-09a3-43f0-96c4-6850033f44d6 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:22.624678 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81793e97-1834-4ad4-955f-0390d0344202 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:22.671184 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15f6f7-f326-489f-932e-45878b7fad30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:22.673001 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15f6f7-f326-489f-932e-45878b7fad30 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:22.673747 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15f6f7-f326-489f-932e-45878b7fad30 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:22.674480 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15f6f7-f326-489f-932e-45878b7fad30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:22.691372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15f6f7-f326-489f-932e-45878b7fad30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:22.695317 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15f6f7-f326-489f-932e-45878b7fad30 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:22.735763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-ef15f6f7-f326-489f-932e-45878b7fad30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:22.735763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:22.735763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:22.735763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:22.735763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:22.735763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:22.735763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:22.735763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:22.735763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:22.735763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:22.735763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:22.735763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:22.735763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:50:22.735763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:50:22.735763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:50:22.737456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:50:22.737456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:22.737456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:22.737456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:22.737456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:22.737456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:22.737456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:22.737456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:22.737456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:22.737456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:22.737456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:22.737456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:22.737456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:22.737456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:22.737456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:22.737456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:22.737456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:22.743546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:50:22.743546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:50:22.743546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:50:22.743546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:50:22.743546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:50:22.743546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:50:22.743546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:22.743546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:22.745747 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:22.749385 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 397/773] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:22.755712 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15f6f7-f326-489f-932e-45878b7fad30 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:22.827112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81793e97-1834-4ad4-955f-0390d0344202 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:22.930230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c90faf-64d8-4569-b45f-c3bf1b471527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:22.931759 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c90faf-64d8-4569-b45f-c3bf1b471527 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:22.932252 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c90faf-64d8-4569-b45f-c3bf1b471527 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:22.932732 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c90faf-64d8-4569-b45f-c3bf1b471527 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:22.933971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c90faf-64d8-4569-b45f-c3bf1b471527 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:22.936720 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c90faf-64d8-4569-b45f-c3bf1b471527 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:22.962465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-b3c90faf-64d8-4569-b45f-c3bf1b471527 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 16 14:50:22.962465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:22.962465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:22.962465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:22.962465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:22.962465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:22.962465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:22.962465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:22.962465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 16 14:50:22.962465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:22.962465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:22.962465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:22.962465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:22.962465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:22.962465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:22.962465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:22.964316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:22.964316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:22.964316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:22.964316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:22.964316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:22.964316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:22.964316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:22.964316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:22.964316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:22.964316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:22.964316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:22.964316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:22.964316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:22.964316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:22.964316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:22.964316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:22.965966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:22.965966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:22.965966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:22.965966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:22.965966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:22.965966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:22.965966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:22.965966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:22.965966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:22.965966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:22.965966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:22.965966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:22.965966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:22.965966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 16 14:50:22.965966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:22.971924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 377/774] 10.222.0.27 () {56 vars in 1129 bytes} [Sat Jul 16 14:50:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:22.975213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c90faf-64d8-4569-b45f-c3bf1b471527 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:23.140994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15f6f7-f326-489f-932e-45878b7fad30 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:23.182658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c90faf-64d8-4569-b45f-c3bf1b471527 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:23.243462 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:23.250927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:23.252569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb8e48-8484-4b40-b710-f322118c2aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:23.263824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb8e48-8484-4b40-b710-f322118c2aff None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:23.263824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb8e48-8484-4b40-b710-f322118c2aff None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:23.263824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb8e48-8484-4b40-b710-f322118c2aff None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:23.263824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb8e48-8484-4b40-b710-f322118c2aff None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:23.263824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb8e48-8484-4b40-b710-f322118c2aff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:23.263824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb8e48-8484-4b40-b710-f322118c2aff None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:23.264591 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8516858-37a2-4f08-9f72-40b3c386ae41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:23.264591 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8516858-37a2-4f08-9f72-40b3c386ae41 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:23.264591 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8516858-37a2-4f08-9f72-40b3c386ae41 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:23.264591 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8516858-37a2-4f08-9f72-40b3c386ae41 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:23.265060 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8516858-37a2-4f08-9f72-40b3c386ae41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:23.265060 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8516858-37a2-4f08-9f72-40b3c386ae41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:23.266241 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8516858-37a2-4f08-9f72-40b3c386ae41 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:23.275099 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 378/775] 10.222.0.27 () {56 vars in 1124 bytes} [Sat Jul 16 14:50:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:23.281337 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb8e48-8484-4b40-b710-f322118c2aff None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:23.296424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 398/776] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:50:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 542 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:23.304840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8516858-37a2-4f08-9f72-40b3c386ae41 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:23.562268 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb8e48-8484-4b40-b710-f322118c2aff None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:23.653766 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcf630-01c4-4e74-b904-e1a9d106867c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:23.655415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcf630-01c4-4e74-b904-e1a9d106867c None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:23.656181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcf630-01c4-4e74-b904-e1a9d106867c None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:23.657639 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcf630-01c4-4e74-b904-e1a9d106867c None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:23.659355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcf630-01c4-4e74-b904-e1a9d106867c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:23.660166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcf630-01c4-4e74-b904-e1a9d106867c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:23.663243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcf630-01c4-4e74-b904-e1a9d106867c None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:23.682879 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 379/777] 10.222.0.27 () {56 vars in 1124 bytes} [Sat Jul 16 14:50:23 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:23.697717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcf630-01c4-4e74-b904-e1a9d106867c None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:23.725392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8516858-37a2-4f08-9f72-40b3c386ae41 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:23.792579 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339bd3a8-a6cb-450b-8170-e278efa1f3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:23.794746 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bd3a8-a6cb-450b-8170-e278efa1f3f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:23.795601 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bd3a8-a6cb-450b-8170-e278efa1f3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:23.796353 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bd3a8-a6cb-450b-8170-e278efa1f3f1 None admin] PATH_INFO: `/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/3f2e241494244edf8d283e8f44192c46/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:23.800709 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339bd3a8-a6cb-450b-8170-e278efa1f3f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=2739f00eb01e46a9acf90ffa360d7d8c, user_id=3f2e241494244edf8d283e8f44192c46, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:23.804647 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339bd3a8-a6cb-450b-8170-e278efa1f3f1 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:23.812865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:23.903571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 399/778] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:50:23 2022] PUT /identity/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/3f2e241494244edf8d283e8f44192c46/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:50:23.907015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74902060-55f1-456d-851e-e101ce2085c9 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:23.907480 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74902060-55f1-456d-851e-e101ce2085c9 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:23.907968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74902060-55f1-456d-851e-e101ce2085c9 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:23.915148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcf630-01c4-4e74-b904-e1a9d106867c None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:23.917694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 400/779] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:23 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:23.924100 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258ddae-e728-4cb8-9290-a7ee1287872a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:23.925703 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258ddae-e728-4cb8-9290-a7ee1287872a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:23.926238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258ddae-e728-4cb8-9290-a7ee1287872a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:23.993115 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-6258ddae-e728-4cb8-9290-a7ee1287872a None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:23.995297 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6258ddae-e728-4cb8-9290-a7ee1287872a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:24.044467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b196062d-ffb4-455f-aa95-5dfac4299dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:24.045635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196062d-ffb4-455f-aa95-5dfac4299dec None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:24.046343 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196062d-ffb4-455f-aa95-5dfac4299dec None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:24.046836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196062d-ffb4-455f-aa95-5dfac4299dec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:24.048402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b196062d-ffb4-455f-aa95-5dfac4299dec None admin] RBAC: Adding query filter params (interface=public, service_id=3a4ffcbab23143eda41e43aa7fa3ac3c, region_id=RegionOne) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:24.048932 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b196062d-ffb4-455f-aa95-5dfac4299dec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:24.053247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b196062d-ffb4-455f-aa95-5dfac4299dec None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:24.065145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 380/780] 10.222.0.27 () {56 vars in 1251 bytes} [Sat Jul 16 14:50:23 2022] GET /identity/v3/endpoints?service_id=3a4ffcbab23143eda41e43aa7fa3ac3c&interface=public®ion_id=RegionOne => generated 194 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:24.067646 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fe666-af0b-4741-a564-165e6dff548e None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:24.068134 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fe666-af0b-4741-a564-165e6dff548e None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:24.068628 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fe666-af0b-4741-a564-165e6dff548e None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:24.076149 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 381/781] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:24.076149 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fe666-af0b-4741-a564-165e6dff548e None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:24.252518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:24.275546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fe666-af0b-4741-a564-165e6dff548e None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:24.278203 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 401/782] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:24.285156 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796dcc7-2036-4adf-b22b-1a1639090a8a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:24.285434 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796dcc7-2036-4adf-b22b-1a1639090a8a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:24.285884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796dcc7-2036-4adf-b22b-1a1639090a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:24.314194 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3796dcc7-2036-4adf-b22b-1a1639090a8a None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:24.315600 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3796dcc7-2036-4adf-b22b-1a1639090a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:24.332264 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:24.356589 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6732b9-38fd-4e8a-a11c-6682dd84d407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:24.358177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6732b9-38fd-4e8a-a11c-6682dd84d407 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:24.358716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6732b9-38fd-4e8a-a11c-6682dd84d407 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:24.359197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6732b9-38fd-4e8a-a11c-6682dd84d407 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:24.360326 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6732b9-38fd-4e8a-a11c-6682dd84d407 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:24.362582 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6732b9-38fd-4e8a-a11c-6682dd84d407 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:24.370991 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 382/783] 10.222.0.27 () {56 vars in 1087 bytes} [Sat Jul 16 14:50:24 2022] GET /identity/v3/services => generated 772 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:24.373131 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6732b9-38fd-4e8a-a11c-6682dd84d407 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:24.481755 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 402/784] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:24.547353 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796dcc7-2036-4adf-b22b-1a1639090a8a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:24.588216 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6732b9-38fd-4e8a-a11c-6682dd84d407 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:24.645794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd22dc-0a10-459c-8451-36cf27312082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:24.647983 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd22dc-0a10-459c-8451-36cf27312082 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:24.648208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd22dc-0a10-459c-8451-36cf27312082 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:24.648833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd22dc-0a10-459c-8451-36cf27312082 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:24.670213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd22dc-0a10-459c-8451-36cf27312082 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:50:24.679114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-0ebd22dc-0a10-459c-8451-36cf27312082 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:50:24.679114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:24.679114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:24.679114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:24.679114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:24.679114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:24.679114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:24.679114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:24.679114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:24.679114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:24.679114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:24.679114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:24.679114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:50:24.679114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:50:24.679114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:50:24.679114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:50:24.680563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:50:24.680563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:50:24.680563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:50:24.680563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:50:24.680563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:24.680563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:24.680563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:24.680563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:24.680563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:24.680563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:24.680563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:24.680563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:24.680563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:24.680563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:24.680563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:24.680563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:24.681398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:24.681398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:24.681398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:24.681398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:24.681398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:50:24.681398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:50:24.681398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:50:24.681398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:50:24.681398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:50:24.681398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:50:24.681398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:50:24.681398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:24.684782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 383/785] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:50:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:24.696328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd22dc-0a10-459c-8451-36cf27312082 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:24.726924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796dcc7-2036-4adf-b22b-1a1639090a8a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:24.745804 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:24.786382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d028fb1-f059-43be-bb22-ce8f6fad6258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:24.788347 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d028fb1-f059-43be-bb22-ce8f6fad6258 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:24.789114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d028fb1-f059-43be-bb22-ce8f6fad6258 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:24.789991 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d028fb1-f059-43be-bb22-ce8f6fad6258 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:24.811915 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d028fb1-f059-43be-bb22-ce8f6fad6258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:50:24.813554 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:24.836018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-9d028fb1-f059-43be-bb22-ce8f6fad6258 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:50:24.836018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:24.836018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:24.836018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:24.836018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:24.836018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:24.836018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:24.836018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:24.836018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:24.836018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:24.836018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:24.836018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:24.836018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:50:24.836018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:50:24.836018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:50:24.836018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:50:24.837474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:50:24.837474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:50:24.837474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:50:24.837474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:50:24.837474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:24.837474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:24.837474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:24.837474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:24.837474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:24.837474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:24.837474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:24.837474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:24.837474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:24.837474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:24.837474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:24.837474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:24.838382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:24.838382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:24.838382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:24.838382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:24.838382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:50:24.838382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:50:24.838382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:50:24.838382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:50:24.838382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:50:24.838382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:50:24.838382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:50:24.838382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:24.844705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 403/786] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:50:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:24.849170 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d028fb1-f059-43be-bb22-ce8f6fad6258 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:24.947947 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd22dc-0a10-459c-8451-36cf27312082 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:25.008404 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954e015-2f03-4176-8498-336b44a9e3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:25.009500 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954e015-2f03-4176-8498-336b44a9e3ee None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:25.009783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954e015-2f03-4176-8498-336b44a9e3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:25.011210 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954e015-2f03-4176-8498-336b44a9e3ee None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:25.012118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954e015-2f03-4176-8498-336b44a9e3ee None admin] RBAC: Adding query filter params (name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:25.012687 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954e015-2f03-4176-8498-336b44a9e3ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:25.014483 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954e015-2f03-4176-8498-336b44a9e3ee None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:25.027923 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 384/787] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:50:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:25.037600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954e015-2f03-4176-8498-336b44a9e3ee None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:25.072843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d028fb1-f059-43be-bb22-ce8f6fad6258 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:25.168314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954e015-2f03-4176-8498-336b44a9e3ee None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:25.170875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:25.187687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af052969-9471-468c-bae0-db759b0db8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:25.188376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af052969-9471-468c-bae0-db759b0db8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:25.188655 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af052969-9471-468c-bae0-db759b0db8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:25.188928 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af052969-9471-468c-bae0-db759b0db8f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:25.189694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af052969-9471-468c-bae0-db759b0db8f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:25.189998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af052969-9471-468c-bae0-db759b0db8f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:25.191462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af052969-9471-468c-bae0-db759b0db8f0 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:25.207563 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 404/788] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:50:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:25.214125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af052969-9471-468c-bae0-db759b0db8f0 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:25.217645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732fd6a-7529-4dcf-810e-969990161a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:25.218453 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732fd6a-7529-4dcf-810e-969990161a0e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:25.218729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732fd6a-7529-4dcf-810e-969990161a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:25.219033 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732fd6a-7529-4dcf-810e-969990161a0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:25.239490 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732fd6a-7529-4dcf-810e-969990161a0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:25.241997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732fd6a-7529-4dcf-810e-969990161a0e None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:25.267769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-0732fd6a-7529-4dcf-810e-969990161a0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:25.267769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:25.267769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:25.267769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:25.267769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:25.267769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:25.267769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:25.267769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:25.267769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:25.267769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:25.267769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:25.267769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:25.267769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:25.267769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:25.267769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:25.268708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:25.268708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:25.268708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:25.268708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:25.268708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:25.268708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:25.268708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:25.268708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:25.268708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:25.268708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:25.268708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:25.268708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:25.268708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:25.268708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:25.268708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:25.268708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:25.268708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:25.269580 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:25.269580 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:25.269580 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:25.269580 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:25.280966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 385/789] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:25.289231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732fd6a-7529-4dcf-810e-969990161a0e None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:25.303298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:25.382943 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af052969-9471-468c-bae0-db759b0db8f0 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:25.429630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732fd6a-7529-4dcf-810e-969990161a0e None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:25.439639 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdbc54-01ed-4210-b23a-29281cba827d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:25.440459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdbc54-01ed-4210-b23a-29281cba827d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:25.440741 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdbc54-01ed-4210-b23a-29281cba827d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:25.441017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdbc54-01ed-4210-b23a-29281cba827d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:25.471610 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fdbc54-01ed-4210-b23a-29281cba827d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:25.474557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fdbc54-01ed-4210-b23a-29281cba827d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:25.475616 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbaf36-97cf-4eb4-8c61-bf06e6a834d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:25.476922 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbaf36-97cf-4eb4-8c61-bf06e6a834d3 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:25.477210 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbaf36-97cf-4eb4-8c61-bf06e6a834d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:25.477482 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbaf36-97cf-4eb4-8c61-bf06e6a834d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:25.478355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dbaf36-97cf-4eb4-8c61-bf06e6a834d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:25.478667 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dbaf36-97cf-4eb4-8c61-bf06e6a834d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:25.480108 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dbaf36-97cf-4eb4-8c61-bf06e6a834d3 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:25.502071 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 386/790] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:25.503968 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd8c44-3b38-43e8-b8cb-11d482fb1994 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:25.504196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-49fdbc54-01ed-4210-b23a-29281cba827d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:25.504196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:25.504196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:25.504196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:25.504196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:25.504196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:25.504196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:25.504196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:25.504196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:25.504196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:25.504196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:25.504196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:25.504196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:25.504196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:25.504196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:25.505235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:25.505235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:25.505235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:25.505235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:25.505235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:25.505235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:25.505235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:25.505235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:25.505235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:25.505235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:25.505235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:25.505235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:25.505235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:25.505235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:25.505235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:25.505235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:25.505235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:25.506220 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:25.506220 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:25.506220 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:25.506220 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:25.506434 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd8c44-3b38-43e8-b8cb-11d482fb1994 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:25.506434 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd8c44-3b38-43e8-b8cb-11d482fb1994 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:25.506555 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 405/791] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:25.509752 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 387/792] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:25.510731 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdbc54-01ed-4210-b23a-29281cba827d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:25.513678 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd8c44-3b38-43e8-b8cb-11d482fb1994 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:25.535632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:25.621179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:25.639250 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd8c44-3b38-43e8-b8cb-11d482fb1994 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:25.641458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fdbc54-01ed-4210-b23a-29281cba827d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:25.678592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50fbc3b-a933-41d0-a114-bd41fae46de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:25.680658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fbc3b-a933-41d0-a114-bd41fae46de3 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:25.680877 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fbc3b-a933-41d0-a114-bd41fae46de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:25.681145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fbc3b-a933-41d0-a114-bd41fae46de3 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:25.682407 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50fbc3b-a933-41d0-a114-bd41fae46de3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:25.682714 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50fbc3b-a933-41d0-a114-bd41fae46de3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:25.684454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50fbc3b-a933-41d0-a114-bd41fae46de3 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:25.697662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce7081-39ff-4b27-96d2-58f27dcb6ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:25.697662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce7081-39ff-4b27-96d2-58f27dcb6ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:25.697662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce7081-39ff-4b27-96d2-58f27dcb6ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:25.697662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce7081-39ff-4b27-96d2-58f27dcb6ac1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:25.707365 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce7081-39ff-4b27-96d2-58f27dcb6ac1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:25.708675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 388/793] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:25.709197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce7081-39ff-4b27-96d2-58f27dcb6ac1 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:25.716585 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680fb1b-9e24-4a49-b7de-a91bc8ff3ea2 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:25.716585 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680fb1b-9e24-4a49-b7de-a91bc8ff3ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:25.716585 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680fb1b-9e24-4a49-b7de-a91bc8ff3ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:25.732736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-d1ce7081-39ff-4b27-96d2-58f27dcb6ac1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 14:50:25.732736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:25.732736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:25.732736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:25.732736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:25.732736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:25.732736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:25.732736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:25.732736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:25.732736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:25.732736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:25.732736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:25.732736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:50:25.732736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:50:25.732736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:50:25.732736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:50:25.733832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:25.733832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:25.733832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:50:25.733832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:25.733832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:50:25.733832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:25.733832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:25.733832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:25.733832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:25.733832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:25.733832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:25.733832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:25.733832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:25.733832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:25.733832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:25.733832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:25.733832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:25.733832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:25.734769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:25.734769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:25.734769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:50:25.734769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:50:25.734769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:50:25.734769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:50:25.734769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:50:25.734769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:50:25.734769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 14:50:25.734769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:25.735602 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 406/794] 10.222.0.27 () {56 vars in 1117 bytes} [Sat Jul 16 14:50:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:25.738757 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce7081-39ff-4b27-96d2-58f27dcb6ac1 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:25.753784 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-c680fb1b-9e24-4a49-b7de-a91bc8ff3ea2 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:25.756053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c680fb1b-9e24-4a49-b7de-a91bc8ff3ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:25.854278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 389/795] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:25.855394 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680fb1b-9e24-4a49-b7de-a91bc8ff3ea2 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:25.863828 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce7081-39ff-4b27-96d2-58f27dcb6ac1 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:25.883238 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:25.909872 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf7bb2-f74b-4d9b-86c9-296dfb6dcc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:25.911504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf7bb2-f74b-4d9b-86c9-296dfb6dcc47 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:25.912005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf7bb2-f74b-4d9b-86c9-296dfb6dcc47 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:25.912478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf7bb2-f74b-4d9b-86c9-296dfb6dcc47 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:25.921991 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf7bb2-f74b-4d9b-86c9-296dfb6dcc47 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:25.923899 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf7bb2-f74b-4d9b-86c9-296dfb6dcc47 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:25.939963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-0adf7bb2-f74b-4d9b-86c9-296dfb6dcc47 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 16 14:50:25.939963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:25.939963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:25.939963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:25.939963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:25.939963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:25.939963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:25.939963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:25.939963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:25.939963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:25.939963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:25.939963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:25.939963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:50:25.939963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:50:25.939963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:50:25.939963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:50:25.940935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:25.940935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:25.940935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:50:25.940935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:25.940935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:50:25.940935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:25.940935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:25.940935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:25.940935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:25.940935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:25.940935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:25.940935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:25.940935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:25.940935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:25.940935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:25.940935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:25.940935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:25.940935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:25.941904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:25.941904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:25.941904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:50:25.941904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:50:25.941904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:50:25.941904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:50:25.941904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:50:25.941904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:50:25.941904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 16 14:50:25.941904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:25.945428 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 407/796] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:50:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:25.946756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf7bb2-f74b-4d9b-86c9-296dfb6dcc47 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:25.958642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:26.003505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680fb1b-9e24-4a49-b7de-a91bc8ff3ea2 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:26.043876 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76117c95-23af-4946-92cf-46694f61ef50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:26.044813 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76117c95-23af-4946-92cf-46694f61ef50 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:26.045237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76117c95-23af-4946-92cf-46694f61ef50 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:26.045651 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76117c95-23af-4946-92cf-46694f61ef50 None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:26.046673 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76117c95-23af-4946-92cf-46694f61ef50 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:26.047280 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76117c95-23af-4946-92cf-46694f61ef50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:26.048925 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76117c95-23af-4946-92cf-46694f61ef50 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:26.060455 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf7bb2-f74b-4d9b-86c9-296dfb6dcc47 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:26.070429 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 390/797] 10.222.0.27 () {56 vars in 1149 bytes} [Sat Jul 16 14:50:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:26.071943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76117c95-23af-4946-92cf-46694f61ef50 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:26.110772 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f85932-92e8-477c-a861-ad2324ad1bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:26.111967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f85932-92e8-477c-a861-ad2324ad1bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:26.112776 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f85932-92e8-477c-a861-ad2324ad1bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:26.113136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f85932-92e8-477c-a861-ad2324ad1bf3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:26.114148 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f85932-92e8-477c-a861-ad2324ad1bf3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:26.115807 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f85932-92e8-477c-a861-ad2324ad1bf3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:26.149808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-80f85932-92e8-477c-a861-ad2324ad1bf3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 16 14:50:26.149808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:26.149808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:26.149808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:26.149808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:26.149808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:26.149808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:26.149808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:26.149808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 16 14:50:26.149808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:26.149808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:26.149808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:26.149808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:26.149808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:26.149808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:26.149808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:26.149808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:26.150867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:26.150867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:26.150867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:26.150867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:26.150867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:26.150867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:26.150867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:26.150867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:26.150867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:26.150867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:26.150867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:26.150867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:26.150867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:26.150867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:26.150867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:26.150867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:26.152053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:26.152053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:26.152053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:26.152053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:26.152053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:26.152053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:26.152053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:26.152053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:26.152053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:26.152053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:26.152053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:26.152053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:26.152053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 16 14:50:26.152053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:26.155035 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 408/798] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:25 2022] GET /identity/v3/services/network => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:26.156143 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f85932-92e8-477c-a861-ad2324ad1bf3 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:26.173711 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:26.230479 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76117c95-23af-4946-92cf-46694f61ef50 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:26.265083 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:26.276270 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4134c-3168-4612-b2f9-27e564606561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:26.277328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4134c-3168-4612-b2f9-27e564606561 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:26.277619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4134c-3168-4612-b2f9-27e564606561 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:26.277898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4134c-3168-4612-b2f9-27e564606561 None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:26.278851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4134c-3168-4612-b2f9-27e564606561 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:26.279490 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4134c-3168-4612-b2f9-27e564606561 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:26.281289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4134c-3168-4612-b2f9-27e564606561 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:26.318684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 391/799] 10.222.0.27 () {56 vars in 1135 bytes} [Sat Jul 16 14:50:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:26.318684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4134c-3168-4612-b2f9-27e564606561 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:26.321008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f85932-92e8-477c-a861-ad2324ad1bf3 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:26.368248 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631539f3-8cb3-4d64-b44b-68d7a48f6207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:26.368953 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631539f3-8cb3-4d64-b44b-68d7a48f6207 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:26.369236 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631539f3-8cb3-4d64-b44b-68d7a48f6207 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:26.369517 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631539f3-8cb3-4d64-b44b-68d7a48f6207 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:26.387655 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631539f3-8cb3-4d64-b44b-68d7a48f6207 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:26.389244 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631539f3-8cb3-4d64-b44b-68d7a48f6207 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:26.438723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-631539f3-8cb3-4d64-b44b-68d7a48f6207 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:26.438723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:26.438723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:26.438723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:26.438723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:26.438723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:26.438723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:26.438723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:26.438723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:26.438723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:26.438723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:26.438723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:26.438723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:26.438723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:26.438723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:26.439927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:26.439927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:26.439927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:26.439927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:26.439927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:26.439927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:26.439927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:26.439927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:26.439927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:26.439927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:26.439927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:26.439927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:26.439927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:26.439927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:26.439927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:26.439927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:26.439927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:26.441043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:26.441043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:26.441043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:26.441043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:26.443190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 409/800] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:26.443190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631539f3-8cb3-4d64-b44b-68d7a48f6207 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:26.502688 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4134c-3168-4612-b2f9-27e564606561 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:26.547956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:26.568991 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e5f53-7073-4ae2-9297-944c2b7507e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:26.570490 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e5f53-7073-4ae2-9297-944c2b7507e5 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:26.571088 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e5f53-7073-4ae2-9297-944c2b7507e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:26.571698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e5f53-7073-4ae2-9297-944c2b7507e5 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:26.573251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e5f53-7073-4ae2-9297-944c2b7507e5 None admin] RBAC: Adding query filter params (name=network) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:26.573797 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e5f53-7073-4ae2-9297-944c2b7507e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:26.577075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e5f53-7073-4ae2-9297-944c2b7507e5 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:26.596787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 392/801] 10.222.0.27 () {56 vars in 1112 bytes} [Sat Jul 16 14:50:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:26.596787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc3b1f-c011-4eef-9e35-fe2b4c562826 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:26.596787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc3b1f-c011-4eef-9e35-fe2b4c562826 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:26.596787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc3b1f-c011-4eef-9e35-fe2b4c562826 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:26.596787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 393/802] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:26.596787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc3b1f-c011-4eef-9e35-fe2b4c562826 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:26.618446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631539f3-8cb3-4d64-b44b-68d7a48f6207 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:26.665619 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:26.688090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2e048-d26b-4292-b7d4-7e999de8ea30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:26.689635 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2e048-d26b-4292-b7d4-7e999de8ea30 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:26.690259 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2e048-d26b-4292-b7d4-7e999de8ea30 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:26.690732 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2e048-d26b-4292-b7d4-7e999de8ea30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:26.713501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2e048-d26b-4292-b7d4-7e999de8ea30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:26.714991 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2e048-d26b-4292-b7d4-7e999de8ea30 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:26.732847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc3b1f-c011-4eef-9e35-fe2b4c562826 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:26.733564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-cab2e048-d26b-4292-b7d4-7e999de8ea30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:26.733564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:26.733564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:26.733564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:26.733564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:26.733564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:26.733564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:26.733564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:26.733564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:26.733564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:26.733564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:26.733564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:26.733564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:26.733564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:26.733564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:26.734478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:26.734478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:26.734478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:26.734478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:26.734478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:26.734478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:26.734478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:26.734478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:26.734478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:26.734478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:26.734478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:26.734478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:26.734478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:26.734478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:26.734478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:26.734478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:26.734478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:26.735948 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:26.735948 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:26.735948 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:26.735948 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:26.739480 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 410/803] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:26.740870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2e048-d26b-4292-b7d4-7e999de8ea30 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:26.798786 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61afaf-f536-4536-8528-a92363437cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:26.799879 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61afaf-f536-4536-8528-a92363437cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:26.800342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61afaf-f536-4536-8528-a92363437cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:26.800817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61afaf-f536-4536-8528-a92363437cc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:26.802143 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61afaf-f536-4536-8528-a92363437cc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:26.802673 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61afaf-f536-4536-8528-a92363437cc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:26.806459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61afaf-f536-4536-8528-a92363437cc0 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:26.831096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 394/804] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:26.832601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897f343-cef0-4937-9739-312f31c45703 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:26.832717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897f343-cef0-4937-9739-312f31c45703 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:26.832939 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897f343-cef0-4937-9739-312f31c45703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:26.863548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-8897f343-cef0-4937-9739-312f31c45703 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:26.865429 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-8897f343-cef0-4937-9739-312f31c45703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:26.894033 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2e048-d26b-4292-b7d4-7e999de8ea30 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:26.942026 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:26.959112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4481473f-6ec2-4716-aee2-fb64c8569ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:26.959816 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481473f-6ec2-4716-aee2-fb64c8569ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:26.960723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481473f-6ec2-4716-aee2-fb64c8569ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:26.961004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481473f-6ec2-4716-aee2-fb64c8569ed9 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:26.962775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4481473f-6ec2-4716-aee2-fb64c8569ed9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:26.962775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4481473f-6ec2-4716-aee2-fb64c8569ed9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:26.963715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4481473f-6ec2-4716-aee2-fb64c8569ed9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:26.970760 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 411/805] 10.222.0.27 () {56 vars in 1112 bytes} [Sat Jul 16 14:50:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:26.971796 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4481473f-6ec2-4716-aee2-fb64c8569ed9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:26.995232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 395/806] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:26.995810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897f343-cef0-4937-9739-312f31c45703 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:27.042223 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:27.104165 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897f343-cef0-4937-9739-312f31c45703 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:27.129130 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4481473f-6ec2-4716-aee2-fb64c8569ed9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:27.142649 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b94b0-ec86-449f-b4df-9e311deba7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:27.143483 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b94b0-ec86-449f-b4df-9e311deba7ca None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:27.143765 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b94b0-ec86-449f-b4df-9e311deba7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:27.144030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b94b0-ec86-449f-b4df-9e311deba7ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:27.154229 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b94b0-ec86-449f-b4df-9e311deba7ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:27.156118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b94b0-ec86-449f-b4df-9e311deba7ca None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:27.170065 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446478b-fb9e-47d3-8876-9a1b1d3e4f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:27.170731 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446478b-fb9e-47d3-8876-9a1b1d3e4f40 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:27.171045 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446478b-fb9e-47d3-8876-9a1b1d3e4f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:27.171204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-723b94b0-ec86-449f-b4df-9e311deba7ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:27.171204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:27.171204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:27.171204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:27.171204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:27.171204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:27.171204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:27.171204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:27.171204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:27.171204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:27.171204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:27.171204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:27.171204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:50:27.171204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:50:27.171204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:50:27.173016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:50:27.173016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:27.173016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:27.173016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:27.173016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:27.173016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:27.173016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:27.173016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:27.173016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:27.173016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:27.173016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:27.173016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:27.173016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:27.173016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:27.173016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:27.173016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:27.173016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:27.174763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:50:27.174763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:50:27.174763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:50:27.174763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:50:27.174763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:50:27.174763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:50:27.174763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:27.174763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:27.175665 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446478b-fb9e-47d3-8876-9a1b1d3e4f40 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:27.175665 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2446478b-fb9e-47d3-8876-9a1b1d3e4f40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:27.175665 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2446478b-fb9e-47d3-8876-9a1b1d3e4f40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:27.176044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 396/807] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:27.176044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b94b0-ec86-449f-b4df-9e311deba7ca None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:27.176288 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2446478b-fb9e-47d3-8876-9a1b1d3e4f40 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:27.189416 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 412/808] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:27.190769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f1fe5-ed15-4bb9-8387-09e6fc4766cd None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:27.191086 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f1fe5-ed15-4bb9-8387-09e6fc4766cd None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:27.191358 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f1fe5-ed15-4bb9-8387-09e6fc4766cd None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:27.192939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 413/809] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:27.193506 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f1fe5-ed15-4bb9-8387-09e6fc4766cd None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:27.227151 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:27.304278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b94b0-ec86-449f-b4df-9e311deba7ca None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:27.304955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:27.309435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f1fe5-ed15-4bb9-8387-09e6fc4766cd None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:27.370132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7001d-2999-436f-9078-86b13024d6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:27.371312 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7001d-2999-436f-9078-86b13024d6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:27.371756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7001d-2999-436f-9078-86b13024d6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:27.372183 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7001d-2999-436f-9078-86b13024d6d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:27.373568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a70b8-7c1c-4713-934e-91b246cc781d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:27.374536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a70b8-7c1c-4713-934e-91b246cc781d None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:27.374972 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a70b8-7c1c-4713-934e-91b246cc781d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:27.375728 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a70b8-7c1c-4713-934e-91b246cc781d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:27.377360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a70b8-7c1c-4713-934e-91b246cc781d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:27.379507 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a70b8-7c1c-4713-934e-91b246cc781d None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:27.391274 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7001d-2999-436f-9078-86b13024d6d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:50:27.405364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-4fb7001d-2999-436f-9078-86b13024d6d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:50:27.405364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:27.405364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:27.405364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:27.405364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:27.405364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:27.405364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:27.405364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:27.405364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:27.405364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:27.405364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:27.405364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:27.405364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:50:27.405364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:50:27.405364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:50:27.405364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:50:27.407162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:50:27.407162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:50:27.407162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:50:27.407162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:50:27.407162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:27.407162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:27.407162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:27.407162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:27.407162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:27.407162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:27.407162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:27.407162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:27.407162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:27.407162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:27.407162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:27.407162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:27.423782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:27.423782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:27.423782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:27.423782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:27.423782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:50:27.423782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:50:27.423782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:50:27.423782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:50:27.423782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:50:27.423782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:50:27.423782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:50:27.423782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:27.423782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 414/810] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:50:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:27.423782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7001d-2999-436f-9078-86b13024d6d0 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:27.505173 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 397/811] 10.222.0.27 () {60 vars in 1144 bytes} [Sat Jul 16 14:50:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:50:27.508267 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca7101-a6e1-40e1-ab6a-cb8eb446d06b None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:27.508826 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca7101-a6e1-40e1-ab6a-cb8eb446d06b None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:27.509351 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca7101-a6e1-40e1-ab6a-cb8eb446d06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:27.545505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-95ca7101-a6e1-40e1-ab6a-cb8eb446d06b None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:27.548949 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-95ca7101-a6e1-40e1-ab6a-cb8eb446d06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:27.608124 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7001d-2999-436f-9078-86b13024d6d0 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:27.690149 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 398/812] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:27.691729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca7101-a6e1-40e1-ab6a-cb8eb446d06b None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:27.700663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4634e8-ebed-4542-90f6-87a86076b4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:27.701404 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4634e8-ebed-4542-90f6-87a86076b4fa None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:27.701693 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4634e8-ebed-4542-90f6-87a86076b4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:27.701972 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4634e8-ebed-4542-90f6-87a86076b4fa None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:27.702811 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4634e8-ebed-4542-90f6-87a86076b4fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:27.703150 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4634e8-ebed-4542-90f6-87a86076b4fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:27.704694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4634e8-ebed-4542-90f6-87a86076b4fa None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:27.725851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:27.735882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 415/813] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:50:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:27.741569 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4634e8-ebed-4542-90f6-87a86076b4fa None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:27.759263 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:27.867695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca7101-a6e1-40e1-ab6a-cb8eb446d06b None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:27.908039 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132e9a8-bf56-4c81-9798-3b6a228e4cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:27.909110 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132e9a8-bf56-4c81-9798-3b6a228e4cda None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:27.909489 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132e9a8-bf56-4c81-9798-3b6a228e4cda None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:27.909855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132e9a8-bf56-4c81-9798-3b6a228e4cda None admin] PATH_INFO: `/v3/projects/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:27.915029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4634e8-ebed-4542-90f6-87a86076b4fa None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:27.918203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132e9a8-bf56-4c81-9798-3b6a228e4cda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:27.920056 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3132e9a8-bf56-4c81-9798-3b6a228e4cda None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:27.935244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-3132e9a8-bf56-4c81-9798-3b6a228e4cda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:27.935244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:27.935244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:27.935244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:27.935244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:27.935244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:27.935244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:27.935244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:27.935244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:27.935244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:27.935244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:27.935244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:27.935244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:50:27.935244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:50:27.935244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:50:27.936998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:50:27.936998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:27.936998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:27.936998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:27.936998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:27.936998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:27.936998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:27.936998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:27.936998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:27.936998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:27.936998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:27.936998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:27.936998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:27.936998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:27.936998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:27.936998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:27.936998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:27.938686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:50:27.938686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:50:27.938686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:50:27.938686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:50:27.938686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:50:27.938686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:50:27.938686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:27.938686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:27.938686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 399/814] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:27.938686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132e9a8-bf56-4c81-9798-3b6a228e4cda None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:27.989583 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92453b7-67f5-4902-9b52-0a8fdb99c7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:27.990647 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92453b7-67f5-4902-9b52-0a8fdb99c7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:27.991721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92453b7-67f5-4902-9b52-0a8fdb99c7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:27.992223 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92453b7-67f5-4902-9b52-0a8fdb99c7f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:27.993694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92453b7-67f5-4902-9b52-0a8fdb99c7f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:27.994253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92453b7-67f5-4902-9b52-0a8fdb99c7f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:27.996936 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92453b7-67f5-4902-9b52-0a8fdb99c7f4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:28.013954 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 416/815] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:50:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:28.015553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92453b7-67f5-4902-9b52-0a8fdb99c7f4 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:28.045574 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:28.070018 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132e9a8-bf56-4c81-9798-3b6a228e4cda None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:28.110114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa2660-ea7e-4f46-bd01-c4c3b8a6f59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:28.111357 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa2660-ea7e-4f46-bd01-c4c3b8a6f59d None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:28.111715 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa2660-ea7e-4f46-bd01-c4c3b8a6f59d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:28.112127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa2660-ea7e-4f46-bd01-c4c3b8a6f59d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:28.112880 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa2660-ea7e-4f46-bd01-c4c3b8a6f59d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:28.114382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa2660-ea7e-4f46-bd01-c4c3b8a6f59d None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:28.128156 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:28.128935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-f3aa2660-ea7e-4f46-bd01-c4c3b8a6f59d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 16 14:50:28.128935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:28.128935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:28.128935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:28.128935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:28.128935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:28.128935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:28.128935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:28.128935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 16 14:50:28.128935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:28.128935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:28.128935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:28.128935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:28.128935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:28.128935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:28.128935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:28.130882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:28.130882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:28.130882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:28.130882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:28.130882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:28.130882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:28.130882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:28.130882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:28.130882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:28.130882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:28.130882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:28.130882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:28.130882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:28.130882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:28.130882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:28.130882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:28.134135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:28.134135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:28.134135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:28.134135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:28.134135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:28.134135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:28.134135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:28.134135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:28.134135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:28.134135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:28.134135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:28.134135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:28.134135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:28.134135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 16 14:50:28.134135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:28.134135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 400/816] 10.222.0.27 () {56 vars in 1129 bytes} [Sat Jul 16 14:50:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:28.135226 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa2660-ea7e-4f46-bd01-c4c3b8a6f59d None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:28.164190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92453b7-67f5-4902-9b52-0a8fdb99c7f4 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:28.206573 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5191e-c67b-43ef-a8cd-df5d1883ce91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:28.207647 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5191e-c67b-43ef-a8cd-df5d1883ce91 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:28.208034 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5191e-c67b-43ef-a8cd-df5d1883ce91 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:28.208423 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5191e-c67b-43ef-a8cd-df5d1883ce91 None admin] PATH_INFO: `/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/a4f499707998403d8f833eb783599466/roles/21119ebd056740048b9ec639ebb20446` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:28.228472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5191e-c67b-43ef-a8cd-df5d1883ce91 None admin] RBAC: Authorizing `identity:create_grant(project_id=2739f00eb01e46a9acf90ffa360d7d8c, user_id=a4f499707998403d8f833eb783599466, role_id=21119ebd056740048b9ec639ebb20446)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:28.230047 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5191e-c67b-43ef-a8cd-df5d1883ce91 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:28.261847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa2660-ea7e-4f46-bd01-c4c3b8a6f59d None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:28.299735 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dde716-ffc2-447f-9cd6-daba1c98c237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:28.300393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dde716-ffc2-447f-9cd6-daba1c98c237 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:28.300673 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dde716-ffc2-447f-9cd6-daba1c98c237 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:28.300946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dde716-ffc2-447f-9cd6-daba1c98c237 None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:28.301733 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dde716-ffc2-447f-9cd6-daba1c98c237 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:28.302035 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dde716-ffc2-447f-9cd6-daba1c98c237 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:28.303506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dde716-ffc2-447f-9cd6-daba1c98c237 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:28.312328 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 417/817] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:50:28 2022] PUT /identity/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/a4f499707998403d8f833eb783599466/roles/21119ebd056740048b9ec639ebb20446 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:50:28.313394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5191e-c67b-43ef-a8cd-df5d1883ce91 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:28.334058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 401/818] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:50:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:28.334058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dde716-ffc2-447f-9cd6-daba1c98c237 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:28.338802 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:28.509964 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dde716-ffc2-447f-9cd6-daba1c98c237 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:28.510281 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5191e-c67b-43ef-a8cd-df5d1883ce91 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:28.543687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:28.554374 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6cbbd-4394-4af1-ae7f-65618d46aff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:28.555990 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6cbbd-4394-4af1-ae7f-65618d46aff5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:28.556281 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6cbbd-4394-4af1-ae7f-65618d46aff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:28.556431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1de21-99a5-4137-8e54-9ae9bb904f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:28.556583 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6cbbd-4394-4af1-ae7f-65618d46aff5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:28.557262 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1de21-99a5-4137-8e54-9ae9bb904f56 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:28.557695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1de21-99a5-4137-8e54-9ae9bb904f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:28.558099 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1de21-99a5-4137-8e54-9ae9bb904f56 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:28.559195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1de21-99a5-4137-8e54-9ae9bb904f56 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:28.559631 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1de21-99a5-4137-8e54-9ae9bb904f56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:28.561358 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1de21-99a5-4137-8e54-9ae9bb904f56 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:28.574940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 402/819] 10.222.0.27 () {56 vars in 1124 bytes} [Sat Jul 16 14:50:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:28.574940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1de21-99a5-4137-8e54-9ae9bb904f56 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:28.577726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6cbbd-4394-4af1-ae7f-65618d46aff5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:28.579232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6cbbd-4394-4af1-ae7f-65618d46aff5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:28.602247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-10b6cbbd-4394-4af1-ae7f-65618d46aff5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:28.602247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:28.602247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:28.602247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:28.602247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:28.602247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:28.602247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:28.602247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:28.602247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:28.602247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:28.602247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:28.602247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:28.602247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:28.602247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:28.602247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:28.604275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:28.604275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:28.604275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:28.604275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:28.604275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:28.604275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:28.604275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:28.604275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:28.604275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:28.604275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:28.604275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:28.604275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:28.604275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:28.604275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:28.604275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:28.604275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:28.604275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:28.606185 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:28.606185 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:28.606185 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:28.606185 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:28.608196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 418/820] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:28.609083 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6cbbd-4394-4af1-ae7f-65618d46aff5 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:28.664338 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:28.711687 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1de21-99a5-4137-8e54-9ae9bb904f56 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:28.755260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68dc23-6487-4f50-b7fe-b8417522dbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:28.756886 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68dc23-6487-4f50-b7fe-b8417522dbca None admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:28.757834 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68dc23-6487-4f50-b7fe-b8417522dbca None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:28.758289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68dc23-6487-4f50-b7fe-b8417522dbca None admin] PATH_INFO: `/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/e1464c83fb294a3d90e5763ef648e834/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:28.760966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68dc23-6487-4f50-b7fe-b8417522dbca None admin] RBAC: Authorizing `identity:create_grant(project_id=2739f00eb01e46a9acf90ffa360d7d8c, user_id=e1464c83fb294a3d90e5763ef648e834, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:28.767369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68dc23-6487-4f50-b7fe-b8417522dbca None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:28.799606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6cbbd-4394-4af1-ae7f-65618d46aff5 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:28.870945 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 403/821] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:50:28 2022] PUT /identity/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/e1464c83fb294a3d90e5763ef648e834/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:50:28.871222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7ab6e-eaac-4d06-aaae-df77414e15cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:28.871222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7ab6e-eaac-4d06-aaae-df77414e15cf None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:28.871548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68dc23-6487-4f50-b7fe-b8417522dbca None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:28.871713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7ab6e-eaac-4d06-aaae-df77414e15cf None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:28.871713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7ab6e-eaac-4d06-aaae-df77414e15cf None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:28.871713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7ab6e-eaac-4d06-aaae-df77414e15cf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:28.871713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7ab6e-eaac-4d06-aaae-df77414e15cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:28.871713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7ab6e-eaac-4d06-aaae-df77414e15cf None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:28.880619 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 419/822] 10.222.0.27 () {56 vars in 1124 bytes} [Sat Jul 16 14:50:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:28.890289 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7ab6e-eaac-4d06-aaae-df77414e15cf None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:28.988317 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:29.119356 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7ab6e-eaac-4d06-aaae-df77414e15cf None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:29.140089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68dc23-6487-4f50-b7fe-b8417522dbca None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:29.185700 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fe89e-7ceb-44cf-b6e9-c93da9390df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:29.187155 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fe89e-7ceb-44cf-b6e9-c93da9390df4 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:29.187885 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fe89e-7ceb-44cf-b6e9-c93da9390df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:29.188671 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fe89e-7ceb-44cf-b6e9-c93da9390df4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:29.190523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367fe89e-7ceb-44cf-b6e9-c93da9390df4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:29.193329 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367fe89e-7ceb-44cf-b6e9-c93da9390df4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:29.195732 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:29.213531 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50713510-e206-420b-9379-3cd33b894017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:29.215005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50713510-e206-420b-9379-3cd33b894017 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:29.215680 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50713510-e206-420b-9379-3cd33b894017 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:29.216324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50713510-e206-420b-9379-3cd33b894017 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:29.217846 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50713510-e206-420b-9379-3cd33b894017 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:29.218570 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50713510-e206-420b-9379-3cd33b894017 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:29.221270 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50713510-e206-420b-9379-3cd33b894017 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:29.245456 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 404/823] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:29.255507 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50713510-e206-420b-9379-3cd33b894017 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:29.304347 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 420/824] 10.222.0.27 () {60 vars in 1144 bytes} [Sat Jul 16 14:50:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:50:29.481122 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50713510-e206-420b-9379-3cd33b894017 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:29.545491 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167b663-df51-4947-8cfa-5b39a4e7d798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:29.546321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167b663-df51-4947-8cfa-5b39a4e7d798 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:29.546598 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167b663-df51-4947-8cfa-5b39a4e7d798 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:29.546870 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167b663-df51-4947-8cfa-5b39a4e7d798 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:29.564727 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167b663-df51-4947-8cfa-5b39a4e7d798 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:29.567754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167b663-df51-4947-8cfa-5b39a4e7d798 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:29.596885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-b167b663-df51-4947-8cfa-5b39a4e7d798 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 16 14:50:29.596885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:29.596885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:29.596885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:29.596885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:29.596885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:29.596885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:29.596885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:29.596885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:29.596885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:29.596885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:29.596885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:29.596885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:50:29.596885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:50:29.596885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:50:29.596885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:50:29.597874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:29.597874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:29.597874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:50:29.597874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:29.597874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:50:29.597874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:29.597874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:29.597874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:29.597874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:29.597874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:29.597874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:29.597874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:29.597874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:29.597874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:29.597874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:29.597874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:29.597874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:29.597874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:29.598805 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:29.598805 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:29.598805 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:50:29.598805 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:50:29.598805 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:50:29.598805 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:50:29.598805 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:50:29.598805 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:50:29.598805 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 16 14:50:29.598805 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:29.603096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 405/825] 10.222.0.27 () {56 vars in 1093 bytes} [Sat Jul 16 14:50:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:29.609013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fe89e-7ceb-44cf-b6e9-c93da9390df4 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:29.798192 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:29.805585 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fe89e-7ceb-44cf-b6e9-c93da9390df4 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:29.869150 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b562e-ac5d-43d3-9841-ff70f901d393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:29.869865 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b562e-ac5d-43d3-9841-ff70f901d393 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:29.870151 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b562e-ac5d-43d3-9841-ff70f901d393 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:29.870424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b562e-ac5d-43d3-9841-ff70f901d393 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:29.871323 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b562e-ac5d-43d3-9841-ff70f901d393 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:29.871633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b562e-ac5d-43d3-9841-ff70f901d393 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:29.873187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b562e-ac5d-43d3-9841-ff70f901d393 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:29.894275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 421/826] 10.222.0.27 () {56 vars in 1133 bytes} [Sat Jul 16 14:50:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:29.906077 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167b663-df51-4947-8cfa-5b39a4e7d798 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:29.957440 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:30.025798 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167b663-df51-4947-8cfa-5b39a4e7d798 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:30.065802 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8e2b2-c0a7-43ad-865e-728f7eb42774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:30.066389 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8e2b2-c0a7-43ad-865e-728f7eb42774 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:30.066673 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8e2b2-c0a7-43ad-865e-728f7eb42774 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:30.066979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8e2b2-c0a7-43ad-865e-728f7eb42774 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:30.150601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8e2b2-c0a7-43ad-865e-728f7eb42774 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:30.152158 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8e2b2-c0a7-43ad-865e-728f7eb42774 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:30.199675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:30.267061 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 406/827] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:50:29 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 14:50:30.273397 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b562e-ac5d-43d3-9841-ff70f901d393 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:30.387381 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b562e-ac5d-43d3-9841-ff70f901d393 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:30.429758 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5af065-615b-4d80-bde4-c560f272997e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:30.430772 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5af065-615b-4d80-bde4-c560f272997e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:30.431275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5af065-615b-4d80-bde4-c560f272997e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:30.431760 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5af065-615b-4d80-bde4-c560f272997e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:30.448875 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5af065-615b-4d80-bde4-c560f272997e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:30.451061 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5af065-615b-4d80-bde4-c560f272997e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:30.485239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-4d5af065-615b-4d80-bde4-c560f272997e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:30.485239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:30.485239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:30.485239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:30.485239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:30.485239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:30.485239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:30.485239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:30.485239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:30.485239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:30.485239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:30.485239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:30.485239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:30.485239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:30.485239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:30.486202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:30.486202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:30.486202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:30.486202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:30.486202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:30.486202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:30.486202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:30.486202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:30.486202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:30.486202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:30.486202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:30.486202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:30.486202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:30.486202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:30.486202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:30.486202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:30.486202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:30.487112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:30.487112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:30.487112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:30.487112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:30.488024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 422/828] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:30.494302 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8e2b2-c0a7-43ad-865e-728f7eb42774 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:30.531307 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5694ab4-9d9f-4cce-bde3-f01fd2cdca95 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:30.531817 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5694ab4-9d9f-4cce-bde3-f01fd2cdca95 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:30.532272 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5694ab4-9d9f-4cce-bde3-f01fd2cdca95 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:30.535126 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 423/829] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:30.545108 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd4044-3b06-4a8b-ba70-693d075c7e75 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:30.545666 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd4044-3b06-4a8b-ba70-693d075c7e75 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:30.546164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd4044-3b06-4a8b-ba70-693d075c7e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:30.578916 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-affd4044-3b06-4a8b-ba70-693d075c7e75 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:30.578916 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-affd4044-3b06-4a8b-ba70-693d075c7e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:30.581127 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:30.611497 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8e2b2-c0a7-43ad-865e-728f7eb42774 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:30.655086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e505a-69fa-4bec-968d-09351a293024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:30.655690 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e505a-69fa-4bec-968d-09351a293024 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:30.655976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e505a-69fa-4bec-968d-09351a293024 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:30.656252 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e505a-69fa-4bec-968d-09351a293024 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:30.656992 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e505a-69fa-4bec-968d-09351a293024 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:30.657302 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e505a-69fa-4bec-968d-09351a293024 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:30.659210 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e505a-69fa-4bec-968d-09351a293024 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:30.675542 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 407/830] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:30.682043 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e505a-69fa-4bec-968d-09351a293024 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:30.685459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:30.723467 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 424/831] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:30.755866 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd4044-3b06-4a8b-ba70-693d075c7e75 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:30.816394 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e505a-69fa-4bec-968d-09351a293024 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:30.861104 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946c8da-e584-4a73-a622-b62a2f1cd4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:30.862565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946c8da-e584-4a73-a622-b62a2f1cd4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:30.862852 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946c8da-e584-4a73-a622-b62a2f1cd4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:30.863457 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946c8da-e584-4a73-a622-b62a2f1cd4e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:30.873863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946c8da-e584-4a73-a622-b62a2f1cd4e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:30.875851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946c8da-e584-4a73-a622-b62a2f1cd4e5 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:30.896236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-e946c8da-e584-4a73-a622-b62a2f1cd4e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:30.896236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:30.896236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:30.896236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:30.896236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:30.896236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:30.896236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:30.896236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:30.896236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:30.896236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:30.896236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:30.896236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:30.896236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:50:30.896236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:50:30.896236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:50:30.897190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:50:30.897190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:30.897190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:30.897190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:30.897190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:30.897190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:30.897190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:30.897190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:30.897190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:30.897190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:30.897190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:30.897190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:30.897190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:30.897190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:30.897190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:30.897190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:30.897190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:30.898626 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:50:30.898626 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:50:30.898626 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:50:30.898626 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:50:30.898626 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:50:30.898626 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:50:30.898626 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:30.898626 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:30.904710 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 408/832] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:30.906473 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946c8da-e584-4a73-a622-b62a2f1cd4e5 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:30.910014 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd4044-3b06-4a8b-ba70-693d075c7e75 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:30.966567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6e168-5a80-4d6b-bc5c-8bac3b28ec64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:30.968204 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6e168-5a80-4d6b-bc5c-8bac3b28ec64 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:30.968846 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6e168-5a80-4d6b-bc5c-8bac3b28ec64 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:30.969335 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6e168-5a80-4d6b-bc5c-8bac3b28ec64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:30.980468 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:30.986694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6e168-5a80-4d6b-bc5c-8bac3b28ec64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:50:30.998383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-71f6e168-5a80-4d6b-bc5c-8bac3b28ec64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:50:30.998383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:30.998383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:30.998383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:30.998383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:30.998383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:30.998383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:30.998383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:30.998383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:30.998383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:30.998383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:30.998383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:30.998383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:50:30.998383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:50:30.998383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:50:30.998383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:50:30.999919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:50:30.999919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:50:30.999919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:50:30.999919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:50:30.999919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:30.999919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:30.999919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:30.999919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:30.999919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:30.999919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:30.999919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:30.999919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:30.999919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:30.999919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:30.999919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:30.999919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:31.001290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:31.001290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:31.001290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:31.001290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:31.001290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:50:31.001290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:50:31.001290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:50:31.001290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:50:31.001290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:50:31.001290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:50:31.001290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:50:31.001290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:31.011462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 425/833] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:50:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:31.013075 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6e168-5a80-4d6b-bc5c-8bac3b28ec64 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:31.032723 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946c8da-e584-4a73-a622-b62a2f1cd4e5 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:31.046714 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:31.086189 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2995f5-1df2-40e0-b3e3-591dcc6f6e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:31.087331 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2995f5-1df2-40e0-b3e3-591dcc6f6e52 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:31.087684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2995f5-1df2-40e0-b3e3-591dcc6f6e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:31.088027 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2995f5-1df2-40e0-b3e3-591dcc6f6e52 None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:31.089249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2995f5-1df2-40e0-b3e3-591dcc6f6e52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:31.089627 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2995f5-1df2-40e0-b3e3-591dcc6f6e52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:31.091682 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2995f5-1df2-40e0-b3e3-591dcc6f6e52 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:31.111008 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 409/834] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:50:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:31.113854 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2995f5-1df2-40e0-b3e3-591dcc6f6e52 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:31.186218 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6e168-5a80-4d6b-bc5c-8bac3b28ec64 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:31.253522 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2995f5-1df2-40e0-b3e3-591dcc6f6e52 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:31.254754 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccab1e4-20ee-44ab-843d-de34bc7ee859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:31.255734 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccab1e4-20ee-44ab-843d-de34bc7ee859 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:31.256739 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccab1e4-20ee-44ab-843d-de34bc7ee859 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:31.257251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccab1e4-20ee-44ab-843d-de34bc7ee859 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:31.259541 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccab1e4-20ee-44ab-843d-de34bc7ee859 None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:31.260164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccab1e4-20ee-44ab-843d-de34bc7ee859 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:31.261809 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccab1e4-20ee-44ab-843d-de34bc7ee859 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:31.274152 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 426/835] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:50:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:31.284713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccab1e4-20ee-44ab-843d-de34bc7ee859 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:31.297259 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:31.314245 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478d609-c227-4ca3-a29a-31657304c6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:31.315408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478d609-c227-4ca3-a29a-31657304c6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:31.316280 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478d609-c227-4ca3-a29a-31657304c6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:31.318062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d478d609-c227-4ca3-a29a-31657304c6c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:31.318753 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478d609-c227-4ca3-a29a-31657304c6c8 None admin] RBAC: Adding query filter params (role.id=0efeff46573247f3b9f1e2dec2968672, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=3f2e241494244edf8d283e8f44192c46) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:31.319436 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478d609-c227-4ca3-a29a-31657304c6c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:31.321934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d478d609-c227-4ca3-a29a-31657304c6c8 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:31.335740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 410/836] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:50:31 2022] GET /identity/v3/role_assignments?user.id=3f2e241494244edf8d283e8f44192c46&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=0efeff46573247f3b9f1e2dec2968672 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:31.391667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:31.449424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccab1e4-20ee-44ab-843d-de34bc7ee859 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:31.491988 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edc901-4ee1-458b-a9be-6cfc440b84a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:31.493269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edc901-4ee1-458b-a9be-6cfc440b84a4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:31.493551 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edc901-4ee1-458b-a9be-6cfc440b84a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:31.493826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edc901-4ee1-458b-a9be-6cfc440b84a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:31.513945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8edc901-4ee1-458b-a9be-6cfc440b84a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:31.515480 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8edc901-4ee1-458b-a9be-6cfc440b84a4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:31.536711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-c8edc901-4ee1-458b-a9be-6cfc440b84a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:31.536711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:31.536711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:31.536711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:31.536711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:31.536711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:31.536711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:31.536711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:31.536711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:31.536711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:31.536711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:31.536711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:31.536711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:31.536711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:31.536711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:31.537692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:31.537692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:31.537692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:31.537692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:31.537692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:31.537692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:31.537692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:31.537692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:31.537692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:31.537692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:31.537692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:31.537692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:31.537692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:31.537692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:31.537692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:31.537692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:31.537692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:31.538630 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:31.538630 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:31.538630 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:31.538630 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:31.541244 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 427/837] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:31.546932 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478d609-c227-4ca3-a29a-31657304c6c8 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:31.674407 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d478d609-c227-4ca3-a29a-31657304c6c8 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:31.715805 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff89a2c-143c-45c3-a4a8-e8c4f93ddafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:31.716451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff89a2c-143c-45c3-a4a8-e8c4f93ddafc None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:31.716735 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff89a2c-143c-45c3-a4a8-e8c4f93ddafc None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:31.717016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff89a2c-143c-45c3-a4a8-e8c4f93ddafc None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:31.717753 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff89a2c-143c-45c3-a4a8-e8c4f93ddafc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:31.718059 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff89a2c-143c-45c3-a4a8-e8c4f93ddafc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:31.719492 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff89a2c-143c-45c3-a4a8-e8c4f93ddafc None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:31.734401 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 411/838] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:31.740082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edc901-4ee1-458b-a9be-6cfc440b84a4 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:31.774460 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8aac04-9ede-42ef-86a0-4fcac762be24 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:31.774749 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8aac04-9ede-42ef-86a0-4fcac762be24 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:31.775063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8aac04-9ede-42ef-86a0-4fcac762be24 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:31.777833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 412/839] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:31.783171 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c274095-2bf6-4459-82bb-b68a5ee7e677 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:31.783456 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c274095-2bf6-4459-82bb-b68a5ee7e677 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:31.784006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c274095-2bf6-4459-82bb-b68a5ee7e677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:31.806929 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-3c274095-2bf6-4459-82bb-b68a5ee7e677 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:31.808747 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3c274095-2bf6-4459-82bb-b68a5ee7e677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:31.866025 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:31.887431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edc901-4ee1-458b-a9be-6cfc440b84a4 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:31.908966 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:31.919002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 413/840] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:31.937356 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c274095-2bf6-4459-82bb-b68a5ee7e677 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:31.940613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056e12d-5423-4bef-a216-df7ef9cdff5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:31.941818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056e12d-5423-4bef-a216-df7ef9cdff5b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:31.942327 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056e12d-5423-4bef-a216-df7ef9cdff5b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:31.942825 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056e12d-5423-4bef-a216-df7ef9cdff5b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:31.953634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056e12d-5423-4bef-a216-df7ef9cdff5b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:31.955782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056e12d-5423-4bef-a216-df7ef9cdff5b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:31.974303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-c056e12d-5423-4bef-a216-df7ef9cdff5b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 14:50:31.974303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:31.974303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:31.974303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:31.974303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:31.974303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:31.974303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:31.974303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:31.974303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:31.974303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:31.974303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:31.974303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:31.974303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:50:31.974303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:50:31.974303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:50:31.974303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:50:31.976136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:31.976136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:31.976136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:50:31.976136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:31.976136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:50:31.976136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:31.976136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:31.976136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:31.976136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:31.976136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:31.976136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:31.976136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:31.976136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:31.976136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:31.976136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:31.976136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:31.976136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:31.976136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:31.977650 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:31.977650 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:31.977650 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:50:31.977650 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:50:31.977650 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:50:31.977650 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:50:31.977650 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:50:31.977650 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:50:31.977650 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 14:50:31.977650 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:31.985053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 428/841] 10.222.0.27 () {56 vars in 1117 bytes} [Sat Jul 16 14:50:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:31.988233 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056e12d-5423-4bef-a216-df7ef9cdff5b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:32.066493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c274095-2bf6-4459-82bb-b68a5ee7e677 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:32.115451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc785fce-6718-4f39-b938-ff51439406e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:32.116980 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc785fce-6718-4f39-b938-ff51439406e2 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:32.117296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc785fce-6718-4f39-b938-ff51439406e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:32.117542 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc785fce-6718-4f39-b938-ff51439406e2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:32.118193 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc785fce-6718-4f39-b938-ff51439406e2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:32.120195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc785fce-6718-4f39-b938-ff51439406e2 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:32.136178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-fc785fce-6718-4f39-b938-ff51439406e2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 16 14:50:32.136178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:32.136178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:32.136178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:32.136178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:32.136178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:32.136178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:32.136178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:32.136178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 16 14:50:32.136178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:32.136178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:32.136178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:32.136178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:32.136178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:32.136178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:32.136178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:32.136178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:32.137249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:32.137249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:32.137249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:32.137249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:32.137249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:32.137249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:32.137249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:32.137249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:32.137249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:32.137249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:32.137249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:32.137249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:32.137249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:32.137249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:32.137249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:32.137249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:32.143204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:32.143204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:32.143204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:32.143204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:32.143204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:32.143204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:32.143204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:32.143204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:32.143204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:32.143204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:32.143204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:32.143204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:32.143204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 16 14:50:32.143204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:32.143204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 414/842] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:50:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:32.143996 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056e12d-5423-4bef-a216-df7ef9cdff5b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:32.145809 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc785fce-6718-4f39-b938-ff51439406e2 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:32.184012 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c5bd8-45ca-4256-b471-94a88f84f0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:32.185107 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c5bd8-45ca-4256-b471-94a88f84f0ac None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:32.185586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c5bd8-45ca-4256-b471-94a88f84f0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:32.186054 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c5bd8-45ca-4256-b471-94a88f84f0ac None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:32.187237 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c5bd8-45ca-4256-b471-94a88f84f0ac None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:32.187755 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c5bd8-45ca-4256-b471-94a88f84f0ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:32.189477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c5bd8-45ca-4256-b471-94a88f84f0ac None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:32.199230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:32.215009 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 429/843] 10.222.0.27 () {56 vars in 1149 bytes} [Sat Jul 16 14:50:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:32.218059 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c5bd8-45ca-4256-b471-94a88f84f0ac None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:32.256064 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:32.282744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc785fce-6718-4f39-b938-ff51439406e2 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:32.329686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65124bd-c178-4327-9120-5db85dc24c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:32.330368 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65124bd-c178-4327-9120-5db85dc24c6b None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:32.330697 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65124bd-c178-4327-9120-5db85dc24c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:32.330994 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65124bd-c178-4327-9120-5db85dc24c6b None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:32.332028 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65124bd-c178-4327-9120-5db85dc24c6b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:32.332337 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65124bd-c178-4327-9120-5db85dc24c6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:32.333932 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65124bd-c178-4327-9120-5db85dc24c6b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:32.345583 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 415/844] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:50:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:32.347300 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c5bd8-45ca-4256-b471-94a88f84f0ac None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:32.348225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65124bd-c178-4327-9120-5db85dc24c6b None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:32.400007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e37c06-ce0a-4a18-b0e6-a0c162f249e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:32.400877 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e37c06-ce0a-4a18-b0e6-a0c162f249e7 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:32.401292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e37c06-ce0a-4a18-b0e6-a0c162f249e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:32.401703 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e37c06-ce0a-4a18-b0e6-a0c162f249e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:32.487165 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65124bd-c178-4327-9120-5db85dc24c6b None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:32.493179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:32.513232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e37c06-ce0a-4a18-b0e6-a0c162f249e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:32.515774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e37c06-ce0a-4a18-b0e6-a0c162f249e7 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:32.544189 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1f2ca-1ce3-4c80-81bf-b7a816beeab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:32.545113 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1f2ca-1ce3-4c80-81bf-b7a816beeab3 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:32.545568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1f2ca-1ce3-4c80-81bf-b7a816beeab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:32.546019 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1f2ca-1ce3-4c80-81bf-b7a816beeab3 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:32.547269 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1f2ca-1ce3-4c80-81bf-b7a816beeab3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:32.547800 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1f2ca-1ce3-4c80-81bf-b7a816beeab3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:32.550141 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1f2ca-1ce3-4c80-81bf-b7a816beeab3 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:32.557328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 416/845] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:50:32 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:32.558468 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91407d8-e5a6-45bd-bf7b-17352c4329ee None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:32.558750 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91407d8-e5a6-45bd-bf7b-17352c4329ee None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:32.559089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91407d8-e5a6-45bd-bf7b-17352c4329ee None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:32.565772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 417/846] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:32.573875 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:32.576871 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ef378-39f9-4c15-b8c0-1c7d75b25c74 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:32.577307 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ef378-39f9-4c15-b8c0-1c7d75b25c74 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:32.577770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ef378-39f9-4c15-b8c0-1c7d75b25c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:32.608294 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-680ef378-39f9-4c15-b8c0-1c7d75b25c74 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:32.610577 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-680ef378-39f9-4c15-b8c0-1c7d75b25c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:32.742916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 418/847] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:32.743173 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 430/848] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:50:32 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 14:50:32.747298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ef378-39f9-4c15-b8c0-1c7d75b25c74 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:32.774786 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e37c06-ce0a-4a18-b0e6-a0c162f249e7 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:32.891760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:32.897091 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ef378-39f9-4c15-b8c0-1c7d75b25c74 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:32.927718 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e37c06-ce0a-4a18-b0e6-a0c162f249e7 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:32.949056 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c482e-ec67-4d5c-99fa-17654bb43a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:32.950459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c482e-ec67-4d5c-99fa-17654bb43a21 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:32.950929 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c482e-ec67-4d5c-99fa-17654bb43a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:32.951648 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c482e-ec67-4d5c-99fa-17654bb43a21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:32.967959 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:32.969866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c482e-ec67-4d5c-99fa-17654bb43a21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:32.971956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c482e-ec67-4d5c-99fa-17654bb43a21 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:32.974989 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c78910-887d-49f7-b5f4-af19f8533180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:32.977004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c78910-887d-49f7-b5f4-af19f8533180 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:32.977581 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c78910-887d-49f7-b5f4-af19f8533180 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:32.977919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c78910-887d-49f7-b5f4-af19f8533180 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:32.992180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c78910-887d-49f7-b5f4-af19f8533180 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:50:32.996716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-0c4c482e-ec67-4d5c-99fa-17654bb43a21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:32.996716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:32.996716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:32.996716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:32.996716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:32.996716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:32.996716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:32.996716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:32.996716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:32.996716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:32.996716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:32.996716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:32.996716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:32.996716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:32.996716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:32.997860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:32.997860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:32.997860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:32.997860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:32.997860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:32.997860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:32.997860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:32.997860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:32.997860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:32.997860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:32.997860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:32.997860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:32.997860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:32.997860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:32.997860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:32.997860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:32.997860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:32.998990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:32.998990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:32.998990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:32.998990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:33.001227 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 419/849] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:33.007641 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c482e-ec67-4d5c-99fa-17654bb43a21 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:33.009773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-d4c78910-887d-49f7-b5f4-af19f8533180 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:50:33.009773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:33.009773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:33.009773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:33.009773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:33.009773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:33.009773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:33.009773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:33.009773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:33.009773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:33.009773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:33.009773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:33.009773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:50:33.009773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:50:33.009773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:50:33.009773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:50:33.010694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:50:33.010694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:50:33.010694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:50:33.010694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:50:33.010694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:33.010694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:33.010694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:33.010694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:33.010694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:33.010694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:33.010694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:33.010694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:33.010694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:33.010694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:33.010694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:33.010694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:33.011626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:33.011626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:33.011626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:33.011626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:33.011626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:50:33.011626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:50:33.011626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:50:33.011626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:50:33.011626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:50:33.011626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:50:33.011626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:50:33.011626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:33.013974 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 431/850] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:50:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:33.021007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c78910-887d-49f7-b5f4-af19f8533180 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:33.148888 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c482e-ec67-4d5c-99fa-17654bb43a21 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:33.162005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c78910-887d-49f7-b5f4-af19f8533180 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:33.204493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd03e3-6ef0-4ac8-861b-12d8b3f5e841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:33.204703 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90486afb-63ac-4804-8e8e-b07ee07030d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:33.205333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90486afb-63ac-4804-8e8e-b07ee07030d8 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:33.205606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90486afb-63ac-4804-8e8e-b07ee07030d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:33.205881 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90486afb-63ac-4804-8e8e-b07ee07030d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:33.206377 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd03e3-6ef0-4ac8-861b-12d8b3f5e841 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:33.206708 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90486afb-63ac-4804-8e8e-b07ee07030d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:33.207068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90486afb-63ac-4804-8e8e-b07ee07030d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:33.207309 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd03e3-6ef0-4ac8-861b-12d8b3f5e841 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:33.207786 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd03e3-6ef0-4ac8-861b-12d8b3f5e841 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:33.208512 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90486afb-63ac-4804-8e8e-b07ee07030d8 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:33.208955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bd03e3-6ef0-4ac8-861b-12d8b3f5e841 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:33.209462 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bd03e3-6ef0-4ac8-861b-12d8b3f5e841 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:33.212851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bd03e3-6ef0-4ac8-861b-12d8b3f5e841 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:33.224387 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 432/851] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:50:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:33.225431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90486afb-63ac-4804-8e8e-b07ee07030d8 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:33.228784 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 420/852] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:33.235978 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd03e3-6ef0-4ac8-861b-12d8b3f5e841 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:33.251634 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:33.335976 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:33.367256 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90486afb-63ac-4804-8e8e-b07ee07030d8 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:33.415180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0873daa-58d8-4e75-96a7-69e312f7f328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:33.416676 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0873daa-58d8-4e75-96a7-69e312f7f328 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:33.417170 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0873daa-58d8-4e75-96a7-69e312f7f328 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:33.417674 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0873daa-58d8-4e75-96a7-69e312f7f328 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:33.419650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd03e3-6ef0-4ac8-861b-12d8b3f5e841 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:33.433735 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0873daa-58d8-4e75-96a7-69e312f7f328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:33.435718 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0873daa-58d8-4e75-96a7-69e312f7f328 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:33.462039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-e0873daa-58d8-4e75-96a7-69e312f7f328 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:33.462039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:33.462039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:33.462039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:33.462039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:33.462039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:33.462039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:33.462039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:33.462039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:33.462039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:33.462039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:33.462039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:33.462039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:33.462039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:33.462039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:33.463089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:33.463089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:33.463089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:33.463089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:33.463089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:33.463089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:33.463089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:33.463089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:33.463089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:33.463089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:33.463089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:33.463089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:33.463089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:33.463089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:33.463089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:33.463089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:33.463089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:33.464020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:33.464020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:33.464020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:33.464020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:33.467067 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 433/853] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:33.475091 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0873daa-58d8-4e75-96a7-69e312f7f328 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:33.490971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49b912-9ce2-42b1-989f-efa700fb2136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:33.492793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49b912-9ce2-42b1-989f-efa700fb2136 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:33.493237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49b912-9ce2-42b1-989f-efa700fb2136 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:33.493665 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49b912-9ce2-42b1-989f-efa700fb2136 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:33.505657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49b912-9ce2-42b1-989f-efa700fb2136 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:33.508260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49b912-9ce2-42b1-989f-efa700fb2136 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:33.534629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-8f49b912-9ce2-42b1-989f-efa700fb2136 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:33.534629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:33.534629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:33.534629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:33.534629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:33.534629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:33.534629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:33.534629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:33.534629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:33.534629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:33.534629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:33.534629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:33.534629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:50:33.534629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:50:33.534629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:50:33.535619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:50:33.535619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:33.535619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:33.535619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:33.535619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:33.535619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:33.535619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:33.535619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:33.535619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:33.535619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:33.535619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:33.535619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:33.535619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:33.535619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:33.535619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:33.535619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:33.535619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:33.536506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:50:33.536506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:50:33.536506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:50:33.536506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:50:33.536506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:50:33.536506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:50:33.536506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:33.536506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:33.543073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 421/854] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:33.547902 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49b912-9ce2-42b1-989f-efa700fb2136 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:33.630613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0873daa-58d8-4e75-96a7-69e312f7f328 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:33.634902 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:33.681524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:33.682485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceebbd7-5314-41c1-b197-a3b7a8254715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:33.684444 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceebbd7-5314-41c1-b197-a3b7a8254715 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:33.684977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceebbd7-5314-41c1-b197-a3b7a8254715 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:33.685460 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceebbd7-5314-41c1-b197-a3b7a8254715 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:33.686432 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceebbd7-5314-41c1-b197-a3b7a8254715 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:33.686985 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceebbd7-5314-41c1-b197-a3b7a8254715 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:33.688635 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceebbd7-5314-41c1-b197-a3b7a8254715 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:33.710400 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 434/855] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:33.712017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceebbd7-5314-41c1-b197-a3b7a8254715 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:33.744078 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49b912-9ce2-42b1-989f-efa700fb2136 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:33.809591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34edcb7-ae70-4eb6-936a-482cc00b8526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:33.810704 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34edcb7-ae70-4eb6-936a-482cc00b8526 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:33.811593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34edcb7-ae70-4eb6-936a-482cc00b8526 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:33.812102 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34edcb7-ae70-4eb6-936a-482cc00b8526 None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:33.813196 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34edcb7-ae70-4eb6-936a-482cc00b8526 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:33.814551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34edcb7-ae70-4eb6-936a-482cc00b8526 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:33.816941 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34edcb7-ae70-4eb6-936a-482cc00b8526 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:33.833931 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 422/856] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:50:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:33.841821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34edcb7-ae70-4eb6-936a-482cc00b8526 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:33.891042 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceebbd7-5314-41c1-b197-a3b7a8254715 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:33.960821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34edcb7-ae70-4eb6-936a-482cc00b8526 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:33.966866 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc613377-8d0f-4528-ad66-a17f2c19bb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:33.968360 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc613377-8d0f-4528-ad66-a17f2c19bb3f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:33.968871 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc613377-8d0f-4528-ad66-a17f2c19bb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:33.969352 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc613377-8d0f-4528-ad66-a17f2c19bb3f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:33.985722 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc613377-8d0f-4528-ad66-a17f2c19bb3f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:33.987986 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc613377-8d0f-4528-ad66-a17f2c19bb3f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:34.007672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-bc613377-8d0f-4528-ad66-a17f2c19bb3f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 16 14:50:34.007672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:34.007672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:34.007672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:34.007672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:34.007672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:34.007672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:34.007672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:34.007672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:34.007672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:34.007672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:34.007672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:34.007672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:50:34.007672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:50:34.007672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:50:34.007672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:50:34.008950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:34.008950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:34.008950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:50:34.008950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:34.008950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:50:34.008950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:34.008950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:34.008950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:34.008950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:34.008950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:34.008950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:34.008950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:34.008950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:34.008950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:34.008950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:34.008950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:34.008950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:34.008950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:34.009976 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:34.009976 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:34.009976 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:50:34.009976 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:50:34.009976 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:50:34.009976 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:50:34.009976 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:50:34.009976 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:50:34.009976 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 16 14:50:34.009976 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:34.017242 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 435/857] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:50:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:34.019387 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc613377-8d0f-4528-ad66-a17f2c19bb3f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:34.025357 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:34.032317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23957e8c-6830-4908-8c37-b49d8d8c72ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:34.033431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23957e8c-6830-4908-8c37-b49d8d8c72ef None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:34.033933 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23957e8c-6830-4908-8c37-b49d8d8c72ef None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:34.034424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23957e8c-6830-4908-8c37-b49d8d8c72ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:34.035581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23957e8c-6830-4908-8c37-b49d8d8c72ef None admin] RBAC: Adding query filter params (role.id=21119ebd056740048b9ec639ebb20446, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=a4f499707998403d8f833eb783599466) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:34.040857 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23957e8c-6830-4908-8c37-b49d8d8c72ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:34.040857 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23957e8c-6830-4908-8c37-b49d8d8c72ef None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:34.055305 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 423/858] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:50:33 2022] GET /identity/v3/role_assignments?user.id=a4f499707998403d8f833eb783599466&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=21119ebd056740048b9ec639ebb20446 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:34.077534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:34.165479 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c513fb-6cb8-4070-b82e-46bca9e957ce None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:34.165950 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c513fb-6cb8-4070-b82e-46bca9e957ce None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:34.166340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c513fb-6cb8-4070-b82e-46bca9e957ce None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:34.168318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 424/859] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:34.177851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f62439-206c-48d9-97ec-e1b7c0bcc93d None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:34.179545 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f62439-206c-48d9-97ec-e1b7c0bcc93d None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:34.180552 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f62439-206c-48d9-97ec-e1b7c0bcc93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:34.212754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-90f62439-206c-48d9-97ec-e1b7c0bcc93d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:34.215464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-90f62439-206c-48d9-97ec-e1b7c0bcc93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:34.234555 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc613377-8d0f-4528-ad66-a17f2c19bb3f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:34.297176 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778be34-f405-4ae6-95b8-efb350faf271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:34.298299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778be34-f405-4ae6-95b8-efb350faf271 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:34.298811 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778be34-f405-4ae6-95b8-efb350faf271 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:34.299302 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778be34-f405-4ae6-95b8-efb350faf271 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:34.300937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778be34-f405-4ae6-95b8-efb350faf271 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:34.301474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778be34-f405-4ae6-95b8-efb350faf271 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:34.305085 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778be34-f405-4ae6-95b8-efb350faf271 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:34.331388 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 436/860] 10.222.0.27 () {56 vars in 1135 bytes} [Sat Jul 16 14:50:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:34.339119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778be34-f405-4ae6-95b8-efb350faf271 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:34.372799 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 425/861] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:34.411977 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f62439-206c-48d9-97ec-e1b7c0bcc93d None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:34.472310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:34.494493 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778be34-f405-4ae6-95b8-efb350faf271 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:34.539030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67ed1e-76ad-4244-ae45-536c591c7289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:34.539719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67ed1e-76ad-4244-ae45-536c591c7289 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:34.539994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67ed1e-76ad-4244-ae45-536c591c7289 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:34.540271 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67ed1e-76ad-4244-ae45-536c591c7289 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:34.606302 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f62439-206c-48d9-97ec-e1b7c0bcc93d None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:34.628466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67ed1e-76ad-4244-ae45-536c591c7289 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:34.630719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67ed1e-76ad-4244-ae45-536c591c7289 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:34.656730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:34.671295 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23e850-8881-443a-bb46-a9f1f63e296f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:34.672141 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23e850-8881-443a-bb46-a9f1f63e296f None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:34.672426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23e850-8881-443a-bb46-a9f1f63e296f None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:34.672706 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23e850-8881-443a-bb46-a9f1f63e296f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:34.673396 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e23e850-8881-443a-bb46-a9f1f63e296f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:34.675082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e23e850-8881-443a-bb46-a9f1f63e296f None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:34.692956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-8e23e850-8881-443a-bb46-a9f1f63e296f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 16 14:50:34.692956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:34.692956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:34.692956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:34.692956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:34.692956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:34.692956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:34.692956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:34.692956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 16 14:50:34.692956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:34.692956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:34.692956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:34.692956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:34.692956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:34.692956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:34.692956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:34.696044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:34.696044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:34.696044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:34.696044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:34.696044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:34.696044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:34.696044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:34.696044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:34.696044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:34.696044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:34.696044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:34.696044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:34.696044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:34.696044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:34.696044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:34.696044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:34.697990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:34.697990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:34.697990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:34.697990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:34.697990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:34.697990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:34.697990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:34.697990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:34.697990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:34.697990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:34.697990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:34.697990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:34.697990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:34.697990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 16 14:50:34.697990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:34.697990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 426/862] 10.222.0.27 () {56 vars in 1132 bytes} [Sat Jul 16 14:50:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:34.703747 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23e850-8881-443a-bb46-a9f1f63e296f None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:34.796372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:34.835579 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 437/863] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:50:34 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 16 14:50:34.841884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67ed1e-76ad-4244-ae45-536c591c7289 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:34.843828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23e850-8881-443a-bb46-a9f1f63e296f None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:34.886307 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4bd56-fc65-4f05-b9fc-c7accf89e3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:34.887021 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4bd56-fc65-4f05-b9fc-c7accf89e3ae None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:34.887305 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4bd56-fc65-4f05-b9fc-c7accf89e3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:34.887592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4bd56-fc65-4f05-b9fc-c7accf89e3ae None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:34.888318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4bd56-fc65-4f05-b9fc-c7accf89e3ae None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:34.888626 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4bd56-fc65-4f05-b9fc-c7accf89e3ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:34.890119 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4bd56-fc65-4f05-b9fc-c7accf89e3ae None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:34.896545 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 427/864] 10.222.0.27 () {56 vars in 1126 bytes} [Sat Jul 16 14:50:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:34.906142 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4bd56-fc65-4f05-b9fc-c7accf89e3ae None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:35.012106 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:35.062046 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67ed1e-76ad-4244-ae45-536c591c7289 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:35.113121 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4bd56-fc65-4f05-b9fc-c7accf89e3ae None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:35.131088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c1525-258a-4626-9abf-e89946ebcd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:35.134186 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c1525-258a-4626-9abf-e89946ebcd48 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:35.134970 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c1525-258a-4626-9abf-e89946ebcd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:35.135808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c1525-258a-4626-9abf-e89946ebcd48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:35.156782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c1525-258a-4626-9abf-e89946ebcd48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:35.160065 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c1525-258a-4626-9abf-e89946ebcd48 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:35.171759 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8ebcf-9478-4f5d-9a0b-fffa7081f501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:35.172435 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8ebcf-9478-4f5d-9a0b-fffa7081f501 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:35.172720 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8ebcf-9478-4f5d-9a0b-fffa7081f501 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:35.173012 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8ebcf-9478-4f5d-9a0b-fffa7081f501 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:35.173711 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8ebcf-9478-4f5d-9a0b-fffa7081f501 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:35.174010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8ebcf-9478-4f5d-9a0b-fffa7081f501 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:35.175503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8ebcf-9478-4f5d-9a0b-fffa7081f501 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:35.181499 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 428/865] 10.222.0.27 () {56 vars in 1126 bytes} [Sat Jul 16 14:50:34 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:35.201519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-035c1525-258a-4626-9abf-e89946ebcd48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:35.201519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:35.201519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:35.201519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:35.201519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:35.201519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:35.201519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:35.201519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:35.201519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:35.201519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:35.201519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:35.201519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:35.201519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:35.201519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:35.201519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:35.203164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:35.203164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:35.203164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:35.203164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:35.203164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:35.203164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:35.203164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:35.203164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:35.203164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:35.203164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:35.203164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:35.203164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:35.203164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:35.203164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:35.203164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:35.203164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:35.203164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:35.204829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:35.204829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:35.204829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:35.204829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:35.208947 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 438/866] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:35.215375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8ebcf-9478-4f5d-9a0b-fffa7081f501 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:35.339004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8ebcf-9478-4f5d-9a0b-fffa7081f501 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:35.363475 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:35.379240 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9bb51-9959-4481-8748-b9a55cdcfd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:35.379935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9bb51-9959-4481-8748-b9a55cdcfd9d None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:35.380221 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9bb51-9959-4481-8748-b9a55cdcfd9d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:35.380505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9bb51-9959-4481-8748-b9a55cdcfd9d None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:35.381311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b9bb51-9959-4481-8748-b9a55cdcfd9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:35.381647 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b9bb51-9959-4481-8748-b9a55cdcfd9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:35.383588 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b9bb51-9959-4481-8748-b9a55cdcfd9d None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:35.398524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 429/867] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:35.405073 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c1525-258a-4626-9abf-e89946ebcd48 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:35.449962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:35.527234 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c1525-258a-4626-9abf-e89946ebcd48 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:35.589509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58985ec8-c316-41b0-9fa9-f440fc4914b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:35.590821 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58985ec8-c316-41b0-9fa9-f440fc4914b9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:35.591368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58985ec8-c316-41b0-9fa9-f440fc4914b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:35.591835 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58985ec8-c316-41b0-9fa9-f440fc4914b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:35.605711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58985ec8-c316-41b0-9fa9-f440fc4914b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:35.608592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58985ec8-c316-41b0-9fa9-f440fc4914b9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:35.639157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-58985ec8-c316-41b0-9fa9-f440fc4914b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:35.639157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:35.639157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:35.639157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:35.639157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:35.639157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:35.639157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:35.639157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:35.639157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:35.639157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:35.639157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:35.639157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:35.639157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:50:35.639157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:50:35.639157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:50:35.640123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:50:35.640123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:35.640123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:35.640123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:35.640123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:35.640123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:35.640123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:35.640123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:35.640123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:35.640123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:35.640123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:35.640123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:35.640123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:35.640123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:35.640123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:35.640123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:35.640123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:35.641029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:50:35.641029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:50:35.641029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:50:35.641029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:50:35.641029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:50:35.641029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:50:35.641029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:35.641029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:35.642543 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 439/868] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:35.648431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9bb51-9959-4481-8748-b9a55cdcfd9d None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:35.799327 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9bb51-9959-4481-8748-b9a55cdcfd9d None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:35.879181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796aa1f-18f0-4f43-b2d8-67c3eac97c12 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:35.879533 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796aa1f-18f0-4f43-b2d8-67c3eac97c12 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:35.879723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796aa1f-18f0-4f43-b2d8-67c3eac97c12 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:35.881389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 440/869] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:35.888388 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697332db-1b52-44f4-a74f-386f34525a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:35.889450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697332db-1b52-44f4-a74f-386f34525a72 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:35.889887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697332db-1b52-44f4-a74f-386f34525a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:35.890222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402a319-177d-4fac-bc0a-b55c74ef1fc0 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:35.890337 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697332db-1b52-44f4-a74f-386f34525a72 None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:35.890676 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402a319-177d-4fac-bc0a-b55c74ef1fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:35.891163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402a319-177d-4fac-bc0a-b55c74ef1fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:35.891740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697332db-1b52-44f4-a74f-386f34525a72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:35.892226 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697332db-1b52-44f4-a74f-386f34525a72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:35.894709 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697332db-1b52-44f4-a74f-386f34525a72 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:35.921736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5402a319-177d-4fac-bc0a-b55c74ef1fc0 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:35.922605 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 430/870] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:50:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:35.924035 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5402a319-177d-4fac-bc0a-b55c74ef1fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:35.928403 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697332db-1b52-44f4-a74f-386f34525a72 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:35.984770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:36.067390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 441/871] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:50:36.084259 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697332db-1b52-44f4-a74f-386f34525a72 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:36.105697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402a319-177d-4fac-bc0a-b55c74ef1fc0 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:36.148961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830fc17-5a2f-43c4-aeaa-222106db741f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:36.149896 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830fc17-5a2f-43c4-aeaa-222106db741f None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:36.150358 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830fc17-5a2f-43c4-aeaa-222106db741f None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:36.150812 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830fc17-5a2f-43c4-aeaa-222106db741f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:36.152464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830fc17-5a2f-43c4-aeaa-222106db741f None admin] RBAC: Adding query filter params (role.id=0efeff46573247f3b9f1e2dec2968672, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=e1464c83fb294a3d90e5763ef648e834) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:36.152985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830fc17-5a2f-43c4-aeaa-222106db741f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:36.153993 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:36.156999 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830fc17-5a2f-43c4-aeaa-222106db741f None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:36.169005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 431/872] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:50:35 2022] GET /identity/v3/role_assignments?user.id=e1464c83fb294a3d90e5763ef648e834&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=0efeff46573247f3b9f1e2dec2968672 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:36.315215 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402a319-177d-4fac-bc0a-b55c74ef1fc0 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:36.370191 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad8665-29bb-43df-82da-d1a9f0d1364a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:36.370877 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad8665-29bb-43df-82da-d1a9f0d1364a None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:36.371194 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad8665-29bb-43df-82da-d1a9f0d1364a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:36.371471 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad8665-29bb-43df-82da-d1a9f0d1364a None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:36.372227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad8665-29bb-43df-82da-d1a9f0d1364a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:36.373627 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad8665-29bb-43df-82da-d1a9f0d1364a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:36.490727 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 442/873] 10.222.0.27 () {60 vars in 1141 bytes} [Sat Jul 16 14:50:36 2022] POST /identity/v3/services => generated 251 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 14:50:36.869760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af0514-e346-4642-aa4f-18b2d2e7d86c None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:36.870564 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af0514-e346-4642-aa4f-18b2d2e7d86c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:36.870564 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af0514-e346-4642-aa4f-18b2d2e7d86c None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:36.872010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 432/874] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:36.878717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410b53f-8e22-49e4-964d-dc8e33e3c33d None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:36.879270 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410b53f-8e22-49e4-964d-dc8e33e3c33d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:36.879779 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410b53f-8e22-49e4-964d-dc8e33e3c33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:36.903768 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-1410b53f-8e22-49e4-964d-dc8e33e3c33d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:36.904978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1410b53f-8e22-49e4-964d-dc8e33e3c33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:37.022048 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 443/875] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:36 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:37.053366 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af0514-e346-4642-aa4f-18b2d2e7d86c None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:37.170971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:37.177213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af0514-e346-4642-aa4f-18b2d2e7d86c None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:37.219390 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac6601-75ff-4d73-8af1-8caa876d8f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:37.220653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac6601-75ff-4d73-8af1-8caa876d8f2f None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:37.221082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac6601-75ff-4d73-8af1-8caa876d8f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:37.221502 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac6601-75ff-4d73-8af1-8caa876d8f2f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:37.233785 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac6601-75ff-4d73-8af1-8caa876d8f2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:50:37.243244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-4aac6601-75ff-4d73-8af1-8caa876d8f2f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 14:50:37.243244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:37.243244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:37.243244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:37.243244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:37.243244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:37.243244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:37.243244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:37.243244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:37.243244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:37.243244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:37.243244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:37.243244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:50:37.243244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:50:37.243244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:50:37.243244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:50:37.244360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:50:37.244360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:50:37.244360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:50:37.244360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:50:37.244360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:37.244360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:37.244360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:37.244360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:37.244360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:37.244360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:37.244360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:37.244360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:37.244360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:37.244360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:37.244360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:37.244360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:37.245332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:37.245332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:37.245332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:37.245332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:37.245332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:50:37.245332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:50:37.245332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:50:37.245332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:50:37.245332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:50:37.245332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:50:37.245332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 14:50:37.245332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:37.247887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 433/876] 10.222.0.27 () {56 vars in 1120 bytes} [Sat Jul 16 14:50:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:37.255611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410b53f-8e22-49e4-964d-dc8e33e3c33d None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:37.266225 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:37.390072 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410b53f-8e22-49e4-964d-dc8e33e3c33d None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:37.427780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02404de6-c67d-490d-bb16-95d8b26e8cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:37.428384 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02404de6-c67d-490d-bb16-95d8b26e8cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:37.429225 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02404de6-c67d-490d-bb16-95d8b26e8cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:37.429608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02404de6-c67d-490d-bb16-95d8b26e8cf5 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:37.430549 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02404de6-c67d-490d-bb16-95d8b26e8cf5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:37.430853 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02404de6-c67d-490d-bb16-95d8b26e8cf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:37.432348 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02404de6-c67d-490d-bb16-95d8b26e8cf5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:37.442038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 444/877] 10.222.0.27 () {56 vars in 1115 bytes} [Sat Jul 16 14:50:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:37.448649 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac6601-75ff-4d73-8af1-8caa876d8f2f None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:37.622031 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac6601-75ff-4d73-8af1-8caa876d8f2f None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:37.667443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4327c3-4d8a-4cb2-9089-bf1899bada7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:37.668558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4327c3-4d8a-4cb2-9089-bf1899bada7e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:37.669000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4327c3-4d8a-4cb2-9089-bf1899bada7e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:37.669440 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4327c3-4d8a-4cb2-9089-bf1899bada7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:37.683151 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4327c3-4d8a-4cb2-9089-bf1899bada7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:37.684873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4327c3-4d8a-4cb2-9089-bf1899bada7e None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:37.713700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-fc4327c3-4d8a-4cb2-9089-bf1899bada7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:37.713700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:37.713700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:37.713700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:37.713700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:37.713700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:37.713700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:37.713700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:37.713700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:37.713700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:37.713700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:37.713700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:37.713700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:37.713700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:37.713700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:37.714869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:37.714869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:37.714869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:37.714869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:37.714869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:37.714869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:37.714869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:37.714869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:37.714869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:37.714869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:37.714869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:37.714869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:37.714869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:37.714869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:37.714869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:37.714869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:37.714869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:37.716024 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:37.716024 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:37.716024 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:37.716024 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:37.716976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 434/878] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:37.721830 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02404de6-c67d-490d-bb16-95d8b26e8cf5 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:37.811854 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:37.854469 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02404de6-c67d-490d-bb16-95d8b26e8cf5 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:37.893553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1906ba4e-d30e-4723-91ab-b91fd53d6cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:37.894227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906ba4e-d30e-4723-91ab-b91fd53d6cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:37.894510 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906ba4e-d30e-4723-91ab-b91fd53d6cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:37.894787 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906ba4e-d30e-4723-91ab-b91fd53d6cf5 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:37.895556 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1906ba4e-d30e-4723-91ab-b91fd53d6cf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:37.895927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1906ba4e-d30e-4723-91ab-b91fd53d6cf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:37.898026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1906ba4e-d30e-4723-91ab-b91fd53d6cf5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:37.913724 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 445/879] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:37.919768 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4327c3-4d8a-4cb2-9089-bf1899bada7e None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:37.937754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:38.042914 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4327c3-4d8a-4cb2-9089-bf1899bada7e None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:38.080885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dad4f8-f42b-4132-bf2c-00f35b496d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:38.081706 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dad4f8-f42b-4132-bf2c-00f35b496d9b None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:38.081987 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dad4f8-f42b-4132-bf2c-00f35b496d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:38.082258 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dad4f8-f42b-4132-bf2c-00f35b496d9b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:38.091484 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dad4f8-f42b-4132-bf2c-00f35b496d9b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:38.092954 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dad4f8-f42b-4132-bf2c-00f35b496d9b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:38.109424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-13dad4f8-f42b-4132-bf2c-00f35b496d9b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 14:50:38.109424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:38.109424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:38.109424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:38.109424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:38.109424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:38.109424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:38.109424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:38.109424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:38.109424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:38.109424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:38.109424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:38.109424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:50:38.109424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:50:38.109424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:50:38.109424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:50:38.110430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:38.110430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:38.110430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:50:38.110430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:38.110430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:50:38.110430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:38.110430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:38.110430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:38.110430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:38.110430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:38.110430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:38.110430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:38.110430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:38.110430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:38.110430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:38.110430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:38.110430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:38.110430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:38.111439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:38.111439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:38.111439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:50:38.111439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:50:38.111439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:50:38.111439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:50:38.111439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:50:38.111439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:50:38.111439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 14:50:38.111439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:38.111439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 435/880] 10.222.0.27 () {56 vars in 1117 bytes} [Sat Jul 16 14:50:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:38.118915 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1906ba4e-d30e-4723-91ab-b91fd53d6cf5 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:38.146145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd681d72-1332-4f6a-b486-26a02971b85d None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:38.146439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd681d72-1332-4f6a-b486-26a02971b85d None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:38.146771 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd681d72-1332-4f6a-b486-26a02971b85d None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:38.149061 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 436/881] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:50:38.154607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc008e-5df1-4bf2-a6bb-e211935bcbf2 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:38.155002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc008e-5df1-4bf2-a6bb-e211935bcbf2 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:38.155190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc008e-5df1-4bf2-a6bb-e211935bcbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:38.172805 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-41dc008e-5df1-4bf2-a6bb-e211935bcbf2 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:38.174081 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-41dc008e-5df1-4bf2-a6bb-e211935bcbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:38.263177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 437/882] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:38 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:50:38.278687 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc008e-5df1-4bf2-a6bb-e211935bcbf2 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:38.281357 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:38.342741 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1906ba4e-d30e-4723-91ab-b91fd53d6cf5 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:38.380608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:38.386460 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db614f-0e72-4dea-b6a5-560e7d27cc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:38.387313 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db614f-0e72-4dea-b6a5-560e7d27cc8d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:38.387697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db614f-0e72-4dea-b6a5-560e7d27cc8d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:38.388072 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db614f-0e72-4dea-b6a5-560e7d27cc8d None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:38.389184 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db614f-0e72-4dea-b6a5-560e7d27cc8d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:38.389807 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db614f-0e72-4dea-b6a5-560e7d27cc8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:38.392620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db614f-0e72-4dea-b6a5-560e7d27cc8d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:38.422132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 446/883] 10.222.0.27 () {56 vars in 1149 bytes} [Sat Jul 16 14:50:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:38.424180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db614f-0e72-4dea-b6a5-560e7d27cc8d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:38.452879 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc008e-5df1-4bf2-a6bb-e211935bcbf2 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:38.505976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e08398-f193-40c5-a911-bed36191b85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:38.506743 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e08398-f193-40c5-a911-bed36191b85b None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:38.507544 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e08398-f193-40c5-a911-bed36191b85b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:38.507819 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e08398-f193-40c5-a911-bed36191b85b None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:38.508504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e08398-f193-40c5-a911-bed36191b85b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:38.509918 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e08398-f193-40c5-a911-bed36191b85b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:38.591130 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 438/884] 10.222.0.27 () {60 vars in 1141 bytes} [Sat Jul 16 14:50:38 2022] POST /identity/v3/services => generated 269 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 14:50:38.612100 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db614f-0e72-4dea-b6a5-560e7d27cc8d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:38.670633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a1dde-0706-4239-81af-5c362054db56 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:38.671712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a1dde-0706-4239-81af-5c362054db56 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:38.672649 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a1dde-0706-4239-81af-5c362054db56 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:38.676596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 439/885] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:38.683329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c4b5f-a6a4-415f-8b33-3f0b4eca0ac7 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:38.683638 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c4b5f-a6a4-415f-8b33-3f0b4eca0ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:38.684028 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c4b5f-a6a4-415f-8b33-3f0b4eca0ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:38.702443 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb050f4a-29e5-4dcd-adfa-738ac3ce8606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:38.704140 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb050f4a-29e5-4dcd-adfa-738ac3ce8606 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:38.704945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb050f4a-29e5-4dcd-adfa-738ac3ce8606 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:38.705814 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb050f4a-29e5-4dcd-adfa-738ac3ce8606 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:38.705991 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-9f1c4b5f-a6a4-415f-8b33-3f0b4eca0ac7 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:38.707108 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-9f1c4b5f-a6a4-415f-8b33-3f0b4eca0ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:38.784089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:38.802023 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 440/886] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:50:38.826036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:38.830801 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c4b5f-a6a4-415f-8b33-3f0b4eca0ac7 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:38.853753 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb050f4a-29e5-4dcd-adfa-738ac3ce8606 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:38.856747 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb050f4a-29e5-4dcd-adfa-738ac3ce8606 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:39.054924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 447/887] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:50:38 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 16 14:50:39.056561 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb050f4a-29e5-4dcd-adfa-738ac3ce8606 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:39.066974 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c4b5f-a6a4-415f-8b33-3f0b4eca0ac7 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:39.124694 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68154cc9-dab7-4e3a-82bf-bab24ecb0054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:39.125470 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68154cc9-dab7-4e3a-82bf-bab24ecb0054 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:39.125751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68154cc9-dab7-4e3a-82bf-bab24ecb0054 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:39.126020 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68154cc9-dab7-4e3a-82bf-bab24ecb0054 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:39.126710 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68154cc9-dab7-4e3a-82bf-bab24ecb0054 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:39.128956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68154cc9-dab7-4e3a-82bf-bab24ecb0054 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:39.145347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-68154cc9-dab7-4e3a-82bf-bab24ecb0054 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 14:50:39.145347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:39.145347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:39.145347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:39.145347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:39.145347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:39.145347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:39.145347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:39.145347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 14:50:39.145347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:39.145347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:39.145347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:39.145347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:39.145347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:39.145347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:39.145347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:39.145347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:39.146413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:39.146413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:39.146413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:39.146413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:39.146413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:39.146413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:39.146413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:39.146413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:39.146413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:39.146413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:39.146413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:39.146413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:39.146413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:39.146413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:39.146413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:39.146413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:39.148375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:39.148375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:39.148375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:39.148375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:39.148375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:39.148375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:39.148375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:39.148375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:39.148375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:39.148375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:39.148375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:39.148375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:39.148375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 14:50:39.148375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:39.148375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 441/888] 10.222.0.27 () {56 vars in 1126 bytes} [Sat Jul 16 14:50:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:39.149693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a8a75-0d7c-4266-b559-f585977d3238 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:39.154954 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a8a75-0d7c-4266-b559-f585977d3238 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:39.154954 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a8a75-0d7c-4266-b559-f585977d3238 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:39.154954 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 442/889] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:39.157358 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a8a75-0d7c-4266-b559-f585977d3238 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:39.217268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:39.240919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb050f4a-29e5-4dcd-adfa-738ac3ce8606 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:39.271802 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:39.295208 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c8437-87ec-4598-8502-989fe6b42a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:39.296664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c8437-87ec-4598-8502-989fe6b42a8b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:39.297143 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c8437-87ec-4598-8502-989fe6b42a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:39.297619 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c8437-87ec-4598-8502-989fe6b42a8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:39.316553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159c8437-87ec-4598-8502-989fe6b42a8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:39.319765 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159c8437-87ec-4598-8502-989fe6b42a8b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:39.327922 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a8a75-0d7c-4266-b559-f585977d3238 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:39.353389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-159c8437-87ec-4598-8502-989fe6b42a8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:39.353389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:39.353389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:39.353389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:39.353389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:39.353389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:39.353389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:39.353389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:39.353389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:39.353389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:39.353389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:39.353389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:39.353389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:39.353389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:39.353389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:39.355608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:39.355608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:39.355608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:39.355608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:39.355608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:39.355608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:39.355608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:39.355608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:39.355608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:39.355608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:39.355608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:39.355608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:39.355608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:39.355608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:39.355608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:39.355608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:39.355608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:39.357359 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:39.357359 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:39.357359 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:39.357359 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:39.362976 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 448/890] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:39.365165 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a8bf4-d63f-4472-bdd3-e70c99118463 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:39.365522 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a8bf4-d63f-4472-bdd3-e70c99118463 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:39.366544 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a8bf4-d63f-4472-bdd3-e70c99118463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:39.369852 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047116c-b20b-4b36-bde1-59552ed86e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:39.370520 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047116c-b20b-4b36-bde1-59552ed86e10 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:39.370799 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047116c-b20b-4b36-bde1-59552ed86e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:39.371099 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047116c-b20b-4b36-bde1-59552ed86e10 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:39.371815 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047116c-b20b-4b36-bde1-59552ed86e10 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:39.372113 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047116c-b20b-4b36-bde1-59552ed86e10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:39.373488 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047116c-b20b-4b36-bde1-59552ed86e10 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:39.387382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 443/891] 10.222.0.27 () {56 vars in 1122 bytes} [Sat Jul 16 14:50:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:39.387382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047116c-b20b-4b36-bde1-59552ed86e10 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:39.396098 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5b4a8bf4-d63f-4472-bdd3-e70c99118463 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:39.399038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5b4a8bf4-d63f-4472-bdd3-e70c99118463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:39.522332 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 449/892] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:39.524876 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a8bf4-d63f-4472-bdd3-e70c99118463 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:39.568479 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047116c-b20b-4b36-bde1-59552ed86e10 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:39.598557 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:39.624827 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235ee2f-ebb7-472f-83a6-20f73f7ac04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:39.625987 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235ee2f-ebb7-472f-83a6-20f73f7ac04a None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:39.626480 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235ee2f-ebb7-472f-83a6-20f73f7ac04a None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:39.627004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235ee2f-ebb7-472f-83a6-20f73f7ac04a None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:39.628436 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235ee2f-ebb7-472f-83a6-20f73f7ac04a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:39.628996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235ee2f-ebb7-472f-83a6-20f73f7ac04a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:39.631652 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235ee2f-ebb7-472f-83a6-20f73f7ac04a None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:39.652690 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:39.662509 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 444/893] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:39.662509 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235ee2f-ebb7-472f-83a6-20f73f7ac04a None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:39.710722 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a8bf4-d63f-4472-bdd3-e70c99118463 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:39.777684 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4298068-8ad3-4fa0-aeb8-1f29fe7ed415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:39.778846 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4298068-8ad3-4fa0-aeb8-1f29fe7ed415 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:39.779388 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4298068-8ad3-4fa0-aeb8-1f29fe7ed415 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:39.779881 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4298068-8ad3-4fa0-aeb8-1f29fe7ed415 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:39.781304 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4298068-8ad3-4fa0-aeb8-1f29fe7ed415 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:39.781846 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4298068-8ad3-4fa0-aeb8-1f29fe7ed415 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:39.784498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4298068-8ad3-4fa0-aeb8-1f29fe7ed415 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:39.798977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 450/894] 10.222.0.27 () {56 vars in 1122 bytes} [Sat Jul 16 14:50:39 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:39.798977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4298068-8ad3-4fa0-aeb8-1f29fe7ed415 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:39.812569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235ee2f-ebb7-472f-83a6-20f73f7ac04a None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:39.877781 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec7344-37d7-4f16-ac6e-dc21883aec53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:39.878849 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec7344-37d7-4f16-ac6e-dc21883aec53 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:39.879281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec7344-37d7-4f16-ac6e-dc21883aec53 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:39.879665 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec7344-37d7-4f16-ac6e-dc21883aec53 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:39.881344 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec7344-37d7-4f16-ac6e-dc21883aec53 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:39.884474 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec7344-37d7-4f16-ac6e-dc21883aec53 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:39.907182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-daec7344-37d7-4f16-ac6e-dc21883aec53 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 16 14:50:39.907182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:39.907182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:39.907182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:39.907182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:39.907182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:39.907182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:39.907182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:39.907182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 16 14:50:39.907182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:39.907182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:39.907182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:39.907182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:39.907182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:39.907182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:39.907182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:39.907182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:39.909174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:39.909174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:39.909174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:39.909174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:39.909174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:39.909174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:39.909174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:39.909174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:39.909174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:39.909174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:39.909174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:39.909174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:39.909174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:39.909174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:39.909174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:39.909174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:39.910450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:39.910450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:39.910450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:39.910450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:39.910450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:39.910450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:39.910450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:39.910450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:39.910450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:39.910450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:39.910450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:39.910450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:39.910450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 16 14:50:39.910450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:39.910450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 445/895] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:50:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:39.916720 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec7344-37d7-4f16-ac6e-dc21883aec53 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:40.007661 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:40.049232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4298068-8ad3-4fa0-aeb8-1f29fe7ed415 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:40.092745 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:40.098218 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd739e-422e-4000-82c5-73809ba8e22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:40.099060 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd739e-422e-4000-82c5-73809ba8e22a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:40.099339 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd739e-422e-4000-82c5-73809ba8e22a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:40.099593 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd739e-422e-4000-82c5-73809ba8e22a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:40.108258 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd739e-422e-4000-82c5-73809ba8e22a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:40.109786 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd739e-422e-4000-82c5-73809ba8e22a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:40.119755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec7344-37d7-4f16-ac6e-dc21883aec53 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:40.125609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-c5bd739e-422e-4000-82c5-73809ba8e22a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:40.125609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:40.125609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:40.125609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:40.125609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:40.125609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:40.125609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:40.125609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:40.125609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:40.125609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:40.125609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:40.125609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:40.125609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:50:40.125609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:50:40.125609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:50:40.127729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:50:40.127729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:40.127729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:40.127729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:40.127729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:40.127729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:40.127729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:40.127729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:40.127729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:40.127729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:40.127729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:40.127729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:40.127729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:40.127729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:40.127729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:40.127729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:40.127729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:40.130222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:50:40.130222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:50:40.130222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:50:40.130222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:50:40.130222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:50:40.130222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:50:40.130222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:40.130222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:40.130222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 451/896] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:40.144906 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd739e-422e-4000-82c5-73809ba8e22a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:40.186578 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cab710-396c-402c-805e-edfd3d4aee80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:40.187636 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cab710-396c-402c-805e-edfd3d4aee80 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:40.188124 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cab710-396c-402c-805e-edfd3d4aee80 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:40.188613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cab710-396c-402c-805e-edfd3d4aee80 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:40.189880 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cab710-396c-402c-805e-edfd3d4aee80 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:40.190401 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cab710-396c-402c-805e-edfd3d4aee80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:40.192978 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cab710-396c-402c-805e-edfd3d4aee80 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:40.202703 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 446/897] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:50:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:40.209120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cab710-396c-402c-805e-edfd3d4aee80 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:40.382965 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd739e-422e-4000-82c5-73809ba8e22a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:40.417513 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cab710-396c-402c-805e-edfd3d4aee80 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:40.439523 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:40.454020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06253657-799a-4191-80f1-ffe47be28cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:40.455222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06253657-799a-4191-80f1-ffe47be28cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:40.455718 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06253657-799a-4191-80f1-ffe47be28cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:40.456224 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06253657-799a-4191-80f1-ffe47be28cc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:40.457765 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06253657-799a-4191-80f1-ffe47be28cc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:40.458329 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06253657-799a-4191-80f1-ffe47be28cc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:40.462967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06253657-799a-4191-80f1-ffe47be28cc9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:40.463085 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ceb5b-bf59-4efd-acbc-14475ac878e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:40.463617 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ceb5b-bf59-4efd-acbc-14475ac878e4 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:40.464034 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ceb5b-bf59-4efd-acbc-14475ac878e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:40.464445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ceb5b-bf59-4efd-acbc-14475ac878e4 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:40.465392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ceb5b-bf59-4efd-acbc-14475ac878e4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:40.465834 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ceb5b-bf59-4efd-acbc-14475ac878e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:40.467485 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ceb5b-bf59-4efd-acbc-14475ac878e4 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:40.478533 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 447/898] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:50:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:40.479852 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ceb5b-bf59-4efd-acbc-14475ac878e4 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:40.487098 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 452/899] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:50:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:40.497057 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06253657-799a-4191-80f1-ffe47be28cc9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:40.603690 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:40.607202 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ceb5b-bf59-4efd-acbc-14475ac878e4 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:40.651512 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e69af-0d59-497c-ab1d-15d0968b7f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:40.652594 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e69af-0d59-497c-ab1d-15d0968b7f2d None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:40.653108 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e69af-0d59-497c-ab1d-15d0968b7f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:40.653596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e69af-0d59-497c-ab1d-15d0968b7f2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:40.654756 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e69af-0d59-497c-ab1d-15d0968b7f2d None admin] RBAC: Adding query filter params (interface=public, service_id=1a29565666fd4714ae3c02e5eb941377, region_id=RegionOne) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:40.655353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e69af-0d59-497c-ab1d-15d0968b7f2d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:40.657023 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e69af-0d59-497c-ab1d-15d0968b7f2d None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:40.664374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 448/900] 10.222.0.27 () {56 vars in 1251 bytes} [Sat Jul 16 14:50:40 2022] GET /identity/v3/endpoints?service_id=1a29565666fd4714ae3c02e5eb941377&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:40.670338 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e69af-0d59-497c-ab1d-15d0968b7f2d None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:40.702734 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06253657-799a-4191-80f1-ffe47be28cc9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:40.714360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:40.775253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315d3d52-2162-4cd3-83a7-410a7b83528a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:40.776884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d3d52-2162-4cd3-83a7-410a7b83528a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:40.777590 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d3d52-2162-4cd3-83a7-410a7b83528a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:40.778320 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315d3d52-2162-4cd3-83a7-410a7b83528a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:40.780353 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d3d52-2162-4cd3-83a7-410a7b83528a None admin] RBAC: Adding query filter params (role.id=e3c9141bd01741b59c8701275be9154c, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=a4f499707998403d8f833eb783599466) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:40.781162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d3d52-2162-4cd3-83a7-410a7b83528a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:40.784467 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315d3d52-2162-4cd3-83a7-410a7b83528a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:40.794216 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e69af-0d59-497c-ab1d-15d0968b7f2d None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:40.797635 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 453/901] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:50:40 2022] GET /identity/v3/role_assignments?user.id=a4f499707998403d8f833eb783599466&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=e3c9141bd01741b59c8701275be9154c => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:40.839852 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d6dd0-4e90-44d6-8ec1-520ee49d6644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:40.840640 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d6dd0-4e90-44d6-8ec1-520ee49d6644 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:40.840984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d6dd0-4e90-44d6-8ec1-520ee49d6644 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:40.841258 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d6dd0-4e90-44d6-8ec1-520ee49d6644 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:40.842106 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d6dd0-4e90-44d6-8ec1-520ee49d6644 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:40.843656 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d6dd0-4e90-44d6-8ec1-520ee49d6644 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:40.851720 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 449/902] 10.222.0.27 () {56 vars in 1087 bytes} [Sat Jul 16 14:50:40 2022] GET /identity/v3/services => generated 1268 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 14:50:41.191066 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e12b7d-8185-4e19-83ce-7850bb3c8922 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:41.191423 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e12b7d-8185-4e19-83ce-7850bb3c8922 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:41.191662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e12b7d-8185-4e19-83ce-7850bb3c8922 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:41.193362 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 454/903] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:41.199476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae653741-7112-4e6c-9cba-7bf459d14782 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:41.200386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae653741-7112-4e6c-9cba-7bf459d14782 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:41.200902 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae653741-7112-4e6c-9cba-7bf459d14782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:41.220679 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-ae653741-7112-4e6c-9cba-7bf459d14782 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:41.222252 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ae653741-7112-4e6c-9cba-7bf459d14782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:41.300794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 450/904] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:41.323276 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e12b7d-8185-4e19-83ce-7850bb3c8922 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:41.465238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e12b7d-8185-4e19-83ce-7850bb3c8922 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:41.484391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:41.508479 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec893a-9897-41e5-ac16-30a2c68dda7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:41.509331 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec893a-9897-41e5-ac16-30a2c68dda7b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:41.509612 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec893a-9897-41e5-ac16-30a2c68dda7b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:41.509887 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec893a-9897-41e5-ac16-30a2c68dda7b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:41.510618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec893a-9897-41e5-ac16-30a2c68dda7b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:41.512120 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec893a-9897-41e5-ac16-30a2c68dda7b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:41.527536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-73ec893a-9897-41e5-ac16-30a2c68dda7b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 16 14:50:41.527536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:41.527536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:41.527536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:41.527536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:41.527536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:41.527536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:41.527536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:41.527536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 16 14:50:41.527536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:41.527536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:41.527536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:41.527536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:41.527536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:41.527536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:41.527536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:41.528711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:41.528711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:41.528711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:41.528711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:41.528711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:41.528711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:41.528711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:41.528711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:41.528711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:41.528711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:41.528711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:41.528711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:41.528711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:41.528711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:41.528711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:41.528711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:41.529562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:41.529562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:41.529562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:41.529562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:41.529562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:41.529562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:41.529562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:41.529562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:41.529562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:41.529562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:41.529562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:41.529562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:41.529562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:41.529562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 16 14:50:41.529562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:41.536627 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 455/905] 10.222.0.27 () {56 vars in 1132 bytes} [Sat Jul 16 14:50:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:41.543814 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae653741-7112-4e6c-9cba-7bf459d14782 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:41.576223 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:41.655995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae653741-7112-4e6c-9cba-7bf459d14782 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:41.697161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d14f76-3ce8-4d66-8bcb-d5140ffd6f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:41.697847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d14f76-3ce8-4d66-8bcb-d5140ffd6f59 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:41.698127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d14f76-3ce8-4d66-8bcb-d5140ffd6f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:41.698402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d14f76-3ce8-4d66-8bcb-d5140ffd6f59 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:41.699158 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d14f76-3ce8-4d66-8bcb-d5140ffd6f59 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:41.699463 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d14f76-3ce8-4d66-8bcb-d5140ffd6f59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:41.700860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d14f76-3ce8-4d66-8bcb-d5140ffd6f59 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:41.706678 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 451/906] 10.222.0.27 () {56 vars in 1126 bytes} [Sat Jul 16 14:50:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:41.714203 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec893a-9897-41e5-ac16-30a2c68dda7b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:41.830314 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec893a-9897-41e5-ac16-30a2c68dda7b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:41.867914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479ecb0-be80-439d-8a1b-adc0a7b3f15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:41.868590 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479ecb0-be80-439d-8a1b-adc0a7b3f15f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:41.868872 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479ecb0-be80-439d-8a1b-adc0a7b3f15f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:41.869148 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6479ecb0-be80-439d-8a1b-adc0a7b3f15f None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:41.869859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479ecb0-be80-439d-8a1b-adc0a7b3f15f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:41.870164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479ecb0-be80-439d-8a1b-adc0a7b3f15f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:41.871578 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6479ecb0-be80-439d-8a1b-adc0a7b3f15f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:41.877487 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 456/907] 10.222.0.27 () {56 vars in 1126 bytes} [Sat Jul 16 14:50:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:41.884243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d14f76-3ce8-4d66-8bcb-d5140ffd6f59 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:41.988966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:42.001485 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d14f76-3ce8-4d66-8bcb-d5140ffd6f59 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:42.067312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948d134-6899-42cc-a6f2-48bcaea2c135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:42.068460 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948d134-6899-42cc-a6f2-48bcaea2c135 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:42.068940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948d134-6899-42cc-a6f2-48bcaea2c135 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:42.070005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948d134-6899-42cc-a6f2-48bcaea2c135 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:42.071243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948d134-6899-42cc-a6f2-48bcaea2c135 None admin] RBAC: Adding query filter params (interface=public, service_id=603ec99a55a9435bb88c9930dd547001, region_id=RegionOne) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:42.071722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948d134-6899-42cc-a6f2-48bcaea2c135 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:42.073905 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948d134-6899-42cc-a6f2-48bcaea2c135 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:42.084571 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 452/908] 10.222.0.27 () {56 vars in 1251 bytes} [Sat Jul 16 14:50:41 2022] GET /identity/v3/endpoints?service_id=603ec99a55a9435bb88c9930dd547001&interface=public®ion_id=RegionOne => generated 194 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:42.091495 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479ecb0-be80-439d-8a1b-adc0a7b3f15f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:42.155460 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:42.220348 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6479ecb0-be80-439d-8a1b-adc0a7b3f15f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:42.261320 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b26901-0092-4121-8de0-d2bbe1d2f94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:42.262004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b26901-0092-4121-8de0-d2bbe1d2f94f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:42.262289 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b26901-0092-4121-8de0-d2bbe1d2f94f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:42.262565 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b26901-0092-4121-8de0-d2bbe1d2f94f None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:42.263290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b26901-0092-4121-8de0-d2bbe1d2f94f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:42.264687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b26901-0092-4121-8de0-d2bbe1d2f94f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:42.271601 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 457/909] 10.222.0.27 () {56 vars in 1087 bytes} [Sat Jul 16 14:50:42 2022] GET /identity/v3/services => generated 1268 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 14:50:43.155637 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d75961-d286-4ca2-8d09-f469a8b1a25a None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:43.156134 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d75961-d286-4ca2-8d09-f469a8b1a25a None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:43.156503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d75961-d286-4ca2-8d09-f469a8b1a25a None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:43.159440 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 453/910] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:43.169039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162eeb3-4c31-40cd-ab5f-bb5278417e24 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:43.169550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162eeb3-4c31-40cd-ab5f-bb5278417e24 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:43.169994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162eeb3-4c31-40cd-ab5f-bb5278417e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:43.200283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-6162eeb3-4c31-40cd-ab5f-bb5278417e24 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:43.202329 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6162eeb3-4c31-40cd-ab5f-bb5278417e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:43.297849 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521cc04-22b8-4113-bd54-0dcd75f1bc6f None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:43.298187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521cc04-22b8-4113-bd54-0dcd75f1bc6f None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:43.298401 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521cc04-22b8-4113-bd54-0dcd75f1bc6f None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:43.300357 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 454/911] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:43.307438 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d238f-0e54-4f68-9b37-05d98a9bde65 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:43.308033 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d238f-0e54-4f68-9b37-05d98a9bde65 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:43.308506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d238f-0e54-4f68-9b37-05d98a9bde65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:43.326730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-7e9d238f-0e54-4f68-9b37-05d98a9bde65 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:43.328352 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7e9d238f-0e54-4f68-9b37-05d98a9bde65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:43.347543 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 458/912] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:43.386567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162eeb3-4c31-40cd-ab5f-bb5278417e24 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:43.411654 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 455/913] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:50:43.432185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d238f-0e54-4f68-9b37-05d98a9bde65 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:43.474587 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:43.568464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:43.588535 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162eeb3-4c31-40cd-ab5f-bb5278417e24 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:43.596127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d238f-0e54-4f68-9b37-05d98a9bde65 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:43.634457 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3592cff-25b9-43bf-b2b4-262888f37175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:43.635767 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3592cff-25b9-43bf-b2b4-262888f37175 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:43.636046 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3592cff-25b9-43bf-b2b4-262888f37175 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:43.636318 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3592cff-25b9-43bf-b2b4-262888f37175 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:43.637060 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3592cff-25b9-43bf-b2b4-262888f37175 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:43.638470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3592cff-25b9-43bf-b2b4-262888f37175 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:43.653242 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733893f-b4ca-4253-8076-4c115fb3507d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:43.654106 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733893f-b4ca-4253-8076-4c115fb3507d None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:43.654433 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733893f-b4ca-4253-8076-4c115fb3507d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:43.656168 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733893f-b4ca-4253-8076-4c115fb3507d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:43.656168 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733893f-b4ca-4253-8076-4c115fb3507d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:43.657045 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733893f-b4ca-4253-8076-4c115fb3507d None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:43.671096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-6733893f-b4ca-4253-8076-4c115fb3507d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 16 14:50:43.671096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:43.671096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:43.671096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:43.671096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:43.671096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:43.671096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:43.671096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:43.671096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 16 14:50:43.671096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:43.671096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:43.671096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:43.671096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:43.671096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:43.671096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:43.671096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:43.671096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:43.672555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:43.672555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:43.672555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:43.672555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:43.672555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:43.672555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:43.672555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:43.672555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:43.672555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:43.672555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:43.672555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:43.672555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:43.672555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:43.672555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:43.672555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:43.672555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:43.674769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:43.674769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:43.674769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:43.674769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:43.674769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:43.674769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:43.674769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:43.674769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:43.674769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:43.674769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:43.674769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:43.674769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:43.674769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 16 14:50:43.674769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:43.674769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 456/914] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:50:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:43.677062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0432c4c-7856-4f0a-a17d-7ac6ee9bb11f None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:43.677062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0432c4c-7856-4f0a-a17d-7ac6ee9bb11f None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:43.677062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0432c4c-7856-4f0a-a17d-7ac6ee9bb11f None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:43.680455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 457/915] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:43.682492 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0432c4c-7856-4f0a-a17d-7ac6ee9bb11f None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:43.720207 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 459/916] 10.222.0.27 () {60 vars in 1141 bytes} [Sat Jul 16 14:50:43 2022] POST /identity/v3/services => generated 241 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 14:50:43.723115 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1f548-32cd-43e5-98cb-d346592fe64e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:43.723729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1f548-32cd-43e5-98cb-d346592fe64e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:43.724304 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1f548-32cd-43e5-98cb-d346592fe64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:43.755392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-75a1f548-32cd-43e5-98cb-d346592fe64e None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:43.758342 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-75a1f548-32cd-43e5-98cb-d346592fe64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:43.924874 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 460/917] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:43.931569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0432c4c-7856-4f0a-a17d-7ac6ee9bb11f None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:43.959755 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1f548-32cd-43e5-98cb-d346592fe64e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:43.976123 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86f7c2-a525-4d49-b6f8-b268995811d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:43.976912 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86f7c2-a525-4d49-b6f8-b268995811d1 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:43.977948 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86f7c2-a525-4d49-b6f8-b268995811d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:43.977948 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86f7c2-a525-4d49-b6f8-b268995811d1 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:43.978256 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86f7c2-a525-4d49-b6f8-b268995811d1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:43.978658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86f7c2-a525-4d49-b6f8-b268995811d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:43.980203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86f7c2-a525-4d49-b6f8-b268995811d1 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:43.986498 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 458/918] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:50:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:43.995872 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86f7c2-a525-4d49-b6f8-b268995811d1 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:44.017035 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:44.026518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:44.185086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86f7c2-a525-4d49-b6f8-b268995811d1 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:44.185686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1f548-32cd-43e5-98cb-d346592fe64e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:44.227267 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2849c-85e4-4647-9f1c-2a0d712e76cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:44.227948 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2849c-85e4-4647-9f1c-2a0d712e76cf None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:44.228230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2849c-85e4-4647-9f1c-2a0d712e76cf None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:44.228562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2849c-85e4-4647-9f1c-2a0d712e76cf None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:44.229449 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2849c-85e4-4647-9f1c-2a0d712e76cf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:44.229761 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2849c-85e4-4647-9f1c-2a0d712e76cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:44.231271 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2849c-85e4-4647-9f1c-2a0d712e76cf None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:44.237763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 459/919] 10.222.0.27 () {56 vars in 1114 bytes} [Sat Jul 16 14:50:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:44.243572 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2849c-85e4-4647-9f1c-2a0d712e76cf None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:44.249699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5360a92-0d6f-49c8-898b-bf264dd217f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:44.251453 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5360a92-0d6f-49c8-898b-bf264dd217f2 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:44.252036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5360a92-0d6f-49c8-898b-bf264dd217f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:44.252648 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5360a92-0d6f-49c8-898b-bf264dd217f2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:44.273410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5360a92-0d6f-49c8-898b-bf264dd217f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:50:44.289498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-e5360a92-0d6f-49c8-898b-bf264dd217f2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 14:50:44.289498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:44.289498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:44.289498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:44.289498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:44.289498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:44.289498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:44.289498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:44.289498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:44.289498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:44.289498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:44.289498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:44.289498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:50:44.289498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:50:44.289498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:50:44.289498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:50:44.290528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:50:44.290528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:50:44.290528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:50:44.290528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:50:44.290528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:44.290528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:44.290528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:44.290528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:44.290528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:44.290528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:44.290528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:44.290528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:44.290528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:44.290528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:44.290528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:44.290528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:44.291431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:44.291431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:44.291431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:44.291431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:44.291431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:50:44.291431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:50:44.291431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:50:44.291431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:50:44.291431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:50:44.291431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:50:44.291431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 14:50:44.291431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:44.299620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 461/920] 10.222.0.27 () {56 vars in 1120 bytes} [Sat Jul 16 14:50:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:44.301257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5360a92-0d6f-49c8-898b-bf264dd217f2 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:44.374367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:44.427560 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2849c-85e4-4647-9f1c-2a0d712e76cf None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:44.485477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:44.501508 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a2a97-566e-4a64-97df-b9c4bede3b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:44.502924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a2a97-566e-4a64-97df-b9c4bede3b97 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:44.503556 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a2a97-566e-4a64-97df-b9c4bede3b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:44.504190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a2a97-566e-4a64-97df-b9c4bede3b97 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:44.505569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a2a97-566e-4a64-97df-b9c4bede3b97 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:44.508845 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a2a97-566e-4a64-97df-b9c4bede3b97 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:44.519029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5360a92-0d6f-49c8-898b-bf264dd217f2 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:44.583539 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be329926-40f5-47dc-845e-cf7fb4882aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:44.585136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be329926-40f5-47dc-845e-cf7fb4882aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:44.585648 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be329926-40f5-47dc-845e-cf7fb4882aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:44.586126 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be329926-40f5-47dc-845e-cf7fb4882aa0 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:44.587904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be329926-40f5-47dc-845e-cf7fb4882aa0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:44.588457 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be329926-40f5-47dc-845e-cf7fb4882aa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:44.591088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be329926-40f5-47dc-845e-cf7fb4882aa0 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:44.604744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 460/921] 10.222.0.27 () {60 vars in 1144 bytes} [Sat Jul 16 14:50:44 2022] POST /identity/v3/endpoints => generated 346 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:50:44.607320 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 462/922] 10.222.0.27 () {56 vars in 1115 bytes} [Sat Jul 16 14:50:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:44.618971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a232f-334d-4d21-93e5-c55ab3575f19 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:44.618971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a232f-334d-4d21-93e5-c55ab3575f19 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:44.618971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a232f-334d-4d21-93e5-c55ab3575f19 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:44.618971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 461/923] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:44 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:44.625012 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be329926-40f5-47dc-845e-cf7fb4882aa0 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:44.629344 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ab3cd-7530-4681-99cc-d1ab10f3e78e None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:44.629801 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ab3cd-7530-4681-99cc-d1ab10f3e78e None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:44.630250 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ab3cd-7530-4681-99cc-d1ab10f3e78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:44.675282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-d41ab3cd-7530-4681-99cc-d1ab10f3e78e None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:44.677333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-d41ab3cd-7530-4681-99cc-d1ab10f3e78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:44.825583 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 462/924] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:44.860582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be329926-40f5-47dc-845e-cf7fb4882aa0 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:44.867572 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ab3cd-7530-4681-99cc-d1ab10f3e78e None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:44.932574 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343c3f9-2cc8-4cca-a3e8-e8693e441519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:44.933270 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343c3f9-2cc8-4cca-a3e8-e8693e441519 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:44.933593 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343c3f9-2cc8-4cca-a3e8-e8693e441519 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:44.933827 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a343c3f9-2cc8-4cca-a3e8-e8693e441519 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:44.949771 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343c3f9-2cc8-4cca-a3e8-e8693e441519 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:44.951290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a343c3f9-2cc8-4cca-a3e8-e8693e441519 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:44.958819 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:44.973540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-a343c3f9-2cc8-4cca-a3e8-e8693e441519 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:44.973540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:44.973540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:44.973540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:44.973540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:44.973540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:44.973540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:44.973540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:44.973540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:44.973540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:44.973540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:44.973540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:44.973540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:44.973540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:44.973540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:44.980706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:44.980706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:44.980706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:44.980706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:44.980706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:44.980706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:44.980706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:44.980706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:44.980706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:44.980706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:44.980706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:44.980706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:44.980706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:44.980706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:44.980706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:44.980706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:44.980706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:44.982367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:44.982367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:44.982367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:44.982367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:44.982367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 463/925] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:44.982367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343c3f9-2cc8-4cca-a3e8-e8693e441519 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:45.000831 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:45.090476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ab3cd-7530-4681-99cc-d1ab10f3e78e None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:45.107591 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a343c3f9-2cc8-4cca-a3e8-e8693e441519 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:45.149859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dca93a-c59a-4806-92bc-ce5cee64c65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:45.150637 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dca93a-c59a-4806-92bc-ce5cee64c65b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:45.150956 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dca93a-c59a-4806-92bc-ce5cee64c65b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:45.151232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dca93a-c59a-4806-92bc-ce5cee64c65b None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:45.152115 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dca93a-c59a-4806-92bc-ce5cee64c65b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:45.152519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dca93a-c59a-4806-92bc-ce5cee64c65b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:45.153954 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dca93a-c59a-4806-92bc-ce5cee64c65b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:45.155841 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98939c7b-b535-4cf6-a441-050a4e4e7752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:45.157492 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98939c7b-b535-4cf6-a441-050a4e4e7752 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:45.158062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98939c7b-b535-4cf6-a441-050a4e4e7752 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:45.158629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98939c7b-b535-4cf6-a441-050a4e4e7752 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:45.159994 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98939c7b-b535-4cf6-a441-050a4e4e7752 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:45.162979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98939c7b-b535-4cf6-a441-050a4e4e7752 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:45.173771 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 464/926] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:45.174866 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dca93a-c59a-4806-92bc-ce5cee64c65b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:45.189096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-98939c7b-b535-4cf6-a441-050a4e4e7752 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 16 14:50:45.189096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:45.189096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:45.189096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:45.189096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:45.189096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:45.189096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:45.189096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:45.189096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 16 14:50:45.189096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:45.189096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:45.189096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:45.189096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:45.189096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:45.189096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:45.189096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:45.192558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:45.192558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:45.192558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:45.192558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:45.192558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:45.192558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:45.192558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:45.192558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:45.192558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:45.192558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:45.192558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:45.192558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:45.192558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:45.192558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:45.192558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:45.192558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:45.201185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:45.201185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:45.201185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:45.201185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:45.201185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:45.201185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:45.201185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:45.201185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:45.201185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:45.201185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:45.201185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:45.201185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:45.201185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:45.201185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 16 14:50:45.201185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:45.201185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 463/927] 10.222.0.27 () {56 vars in 1132 bytes} [Sat Jul 16 14:50:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:45.214940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98939c7b-b535-4cf6-a441-050a4e4e7752 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:45.245658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:45.296550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dca93a-c59a-4806-92bc-ce5cee64c65b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:45.357755 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875b6fe-837c-4f76-8a57-2c84975cd07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:45.359146 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875b6fe-837c-4f76-8a57-2c84975cd07c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:45.359613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875b6fe-837c-4f76-8a57-2c84975cd07c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:45.360063 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875b6fe-837c-4f76-8a57-2c84975cd07c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:45.378380 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e875b6fe-837c-4f76-8a57-2c84975cd07c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:45.380744 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e875b6fe-837c-4f76-8a57-2c84975cd07c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:45.415967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-e875b6fe-837c-4f76-8a57-2c84975cd07c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 14:50:45.415967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:45.415967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:45.415967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:45.415967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:45.415967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:45.415967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:45.415967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:45.415967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:45.415967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:45.415967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:45.415967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:45.415967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:50:45.415967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:50:45.415967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:50:45.415967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:50:45.418172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:45.418172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:45.418172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:50:45.418172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:45.418172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:50:45.418172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:45.418172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:45.418172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:45.418172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:45.418172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:45.418172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:45.418172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:45.418172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:45.418172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:45.418172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:45.418172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:45.418172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:45.418172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:45.419939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:45.419939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:45.419939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:50:45.419939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:50:45.419939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:50:45.419939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:50:45.419939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:50:45.419939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:50:45.419939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 14:50:45.419939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:45.419939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 465/928] 10.222.0.27 () {56 vars in 1117 bytes} [Sat Jul 16 14:50:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:45.427116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98939c7b-b535-4cf6-a441-050a4e4e7752 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:45.427438 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875b6fe-837c-4f76-8a57-2c84975cd07c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:45.477110 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:45.493220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbb87f-9cd8-4d7c-9310-a41ef0750c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:45.494327 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbb87f-9cd8-4d7c-9310-a41ef0750c5e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:45.494821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbb87f-9cd8-4d7c-9310-a41ef0750c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:45.495416 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbb87f-9cd8-4d7c-9310-a41ef0750c5e None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:45.496377 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbb87f-9cd8-4d7c-9310-a41ef0750c5e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:45.496917 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbb87f-9cd8-4d7c-9310-a41ef0750c5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:45.498690 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbb87f-9cd8-4d7c-9310-a41ef0750c5e None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:45.512174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 464/929] 10.222.0.27 () {56 vars in 1126 bytes} [Sat Jul 16 14:50:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:45.513964 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbb87f-9cd8-4d7c-9310-a41ef0750c5e None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:45.599610 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875b6fe-837c-4f76-8a57-2c84975cd07c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:45.636408 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:45.676966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbb87f-9cd8-4d7c-9310-a41ef0750c5e None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:45.685776 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061a3f4-b393-45a7-9450-6a936c6e741a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:45.686969 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061a3f4-b393-45a7-9450-6a936c6e741a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:45.688419 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061a3f4-b393-45a7-9450-6a936c6e741a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:45.688930 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061a3f4-b393-45a7-9450-6a936c6e741a None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:45.690535 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061a3f4-b393-45a7-9450-6a936c6e741a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:45.691125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061a3f4-b393-45a7-9450-6a936c6e741a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:45.696425 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b061a3f4-b393-45a7-9450-6a936c6e741a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:45.724711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 466/930] 10.222.0.27 () {56 vars in 1149 bytes} [Sat Jul 16 14:50:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:45.735360 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061a3f4-b393-45a7-9450-6a936c6e741a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:45.746127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f0884-a8ad-4bdb-b8b3-0cd45809ab05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:45.748317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f0884-a8ad-4bdb-b8b3-0cd45809ab05 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:45.748803 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f0884-a8ad-4bdb-b8b3-0cd45809ab05 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:45.749296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f0884-a8ad-4bdb-b8b3-0cd45809ab05 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:45.750678 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f0884-a8ad-4bdb-b8b3-0cd45809ab05 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:45.752022 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f0884-a8ad-4bdb-b8b3-0cd45809ab05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:45.755644 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f0884-a8ad-4bdb-b8b3-0cd45809ab05 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:45.770303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 465/931] 10.222.0.27 () {56 vars in 1126 bytes} [Sat Jul 16 14:50:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:45.778573 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f0884-a8ad-4bdb-b8b3-0cd45809ab05 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:45.879878 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b061a3f4-b393-45a7-9450-6a936c6e741a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:45.919941 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:45.932813 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff42a9-8f12-41d0-9652-bd1f20fa1338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:45.939094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff42a9-8f12-41d0-9652-bd1f20fa1338 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:45.939094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff42a9-8f12-41d0-9652-bd1f20fa1338 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:45.939094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff42a9-8f12-41d0-9652-bd1f20fa1338 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:45.953644 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff42a9-8f12-41d0-9652-bd1f20fa1338 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:45.958507 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff42a9-8f12-41d0-9652-bd1f20fa1338 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:45.999526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f0884-a8ad-4bdb-b8b3-0cd45809ab05 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:46.003526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-69ff42a9-8f12-41d0-9652-bd1f20fa1338 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:46.003526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:46.003526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:46.003526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:46.003526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:46.003526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:46.003526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:46.003526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:46.003526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:46.003526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:46.003526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:46.003526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:46.003526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:46.003526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:46.003526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:46.005041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:46.005041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:46.005041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:46.005041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:46.005041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:46.005041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:46.005041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:46.005041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:46.005041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:46.005041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:46.005041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:46.005041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:46.005041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:46.005041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:46.005041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:46.005041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:46.005041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:46.007523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:46.007523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:46.007523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:46.007523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:46.007523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 467/932] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:46.012657 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462ea06-5096-41a8-8350-91673674d457 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:46.013037 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462ea06-5096-41a8-8350-91673674d457 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:46.013277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462ea06-5096-41a8-8350-91673674d457 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:46.015034 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 468/933] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:46.020363 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462ea06-5096-41a8-8350-91673674d457 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:46.043285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff363d6-32ce-431e-a4cc-a81ea04227f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:46.043968 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff363d6-32ce-431e-a4cc-a81ea04227f8 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:46.044251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff363d6-32ce-431e-a4cc-a81ea04227f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:46.044519 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff363d6-32ce-431e-a4cc-a81ea04227f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:46.045289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff363d6-32ce-431e-a4cc-a81ea04227f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:46.046941 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff363d6-32ce-431e-a4cc-a81ea04227f8 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:46.060979 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:46.146677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 466/934] 10.222.0.27 () {60 vars in 1144 bytes} [Sat Jul 16 14:50:45 2022] POST /identity/v3/endpoints => generated 347 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 14:50:46.146677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fdad5-b33f-45bf-a0c9-99a0d8c91ff3 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:46.146677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fdad5-b33f-45bf-a0c9-99a0d8c91ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:46.146677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fdad5-b33f-45bf-a0c9-99a0d8c91ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:46.168687 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-ef3fdad5-b33f-45bf-a0c9-99a0d8c91ff3 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:46.170798 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ef3fdad5-b33f-45bf-a0c9-99a0d8c91ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:46.175350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462ea06-5096-41a8-8350-91673674d457 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:46.243923 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf5b2f-ce6f-4b1b-8118-0e12940f8308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:46.245018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf5b2f-ce6f-4b1b-8118-0e12940f8308 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:46.245592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf5b2f-ce6f-4b1b-8118-0e12940f8308 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:46.246078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf5b2f-ce6f-4b1b-8118-0e12940f8308 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:46.247599 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf5b2f-ce6f-4b1b-8118-0e12940f8308 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:46.248289 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf5b2f-ce6f-4b1b-8118-0e12940f8308 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:46.250805 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf5b2f-ce6f-4b1b-8118-0e12940f8308 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:46.279231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 469/935] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:46.283715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf5b2f-ce6f-4b1b-8118-0e12940f8308 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:46.311337 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 467/936] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:46.348909 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3fdad5-b33f-45bf-a0c9-99a0d8c91ff3 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:46.407176 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:46.418210 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:46.433429 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf5b2f-ce6f-4b1b-8118-0e12940f8308 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:46.497713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d246739-f7d3-4fa4-958a-d310ba6e7637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:46.500305 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d246739-f7d3-4fa4-958a-d310ba6e7637 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:46.501080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d246739-f7d3-4fa4-958a-d310ba6e7637 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:46.501834 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d246739-f7d3-4fa4-958a-d310ba6e7637 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:46.519413 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d246739-f7d3-4fa4-958a-d310ba6e7637 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:46.523920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d246739-f7d3-4fa4-958a-d310ba6e7637 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:46.542633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3fdad5-b33f-45bf-a0c9-99a0d8c91ff3 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:46.559592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-0d246739-f7d3-4fa4-958a-d310ba6e7637 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:46.559592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:46.559592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:46.559592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:46.559592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:46.559592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:46.559592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:46.559592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:46.559592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:46.559592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:46.559592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:46.559592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:46.559592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:50:46.559592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:50:46.559592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:50:46.561298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:50:46.561298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:46.561298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:46.561298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:46.561298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:46.561298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:46.561298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:46.561298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:46.561298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:46.561298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:46.561298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:46.561298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:46.561298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:46.561298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:46.561298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:46.561298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:46.561298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:46.563079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:50:46.563079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:50:46.563079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:50:46.563079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:50:46.563079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:50:46.563079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:50:46.563079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:46.563079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:46.563079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 470/937] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:46.572153 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d246739-f7d3-4fa4-958a-d310ba6e7637 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:46.632522 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c06228-b801-4950-b149-e81590cbd0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:46.635064 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c06228-b801-4950-b149-e81590cbd0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:46.636142 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c06228-b801-4950-b149-e81590cbd0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:46.636910 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c06228-b801-4950-b149-e81590cbd0f7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:46.639211 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c06228-b801-4950-b149-e81590cbd0f7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:46.641817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c06228-b801-4950-b149-e81590cbd0f7 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:46.669198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-b2c06228-b801-4950-b149-e81590cbd0f7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 14:50:46.669198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:46.669198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:46.669198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:46.669198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:46.669198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:46.669198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:46.669198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:46.669198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 14:50:46.669198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:46.669198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:46.669198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:46.669198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:46.669198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:46.669198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:46.669198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:46.669198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:46.671116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:46.671116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:46.671116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:46.671116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:46.671116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:46.671116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:46.671116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:46.671116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:46.671116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:46.671116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:46.671116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:46.671116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:46.671116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:46.671116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:46.671116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:46.671116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:46.672754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:46.672754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:46.672754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:46.672754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:46.672754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:46.672754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:46.672754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:46.672754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:46.672754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:46.672754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:46.672754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:46.672754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:46.672754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 14:50:46.672754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:46.672754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 468/938] 10.222.0.27 () {56 vars in 1126 bytes} [Sat Jul 16 14:50:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:46.684702 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c06228-b801-4950-b149-e81590cbd0f7 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:46.806761 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d246739-f7d3-4fa4-958a-d310ba6e7637 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:46.878116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:46.887825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c06228-b801-4950-b149-e81590cbd0f7 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:46.889001 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d6079-1089-40cd-aaee-90521eec3e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:46.890512 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d6079-1089-40cd-aaee-90521eec3e63 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:46.891409 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d6079-1089-40cd-aaee-90521eec3e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:46.891996 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d6079-1089-40cd-aaee-90521eec3e63 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:46.894699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d6079-1089-40cd-aaee-90521eec3e63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:46.895418 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d6079-1089-40cd-aaee-90521eec3e63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:46.899363 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d6079-1089-40cd-aaee-90521eec3e63 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:46.939663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 471/939] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:50:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:46.943389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d6079-1089-40cd-aaee-90521eec3e63 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:46.951108 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:46.965629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e3c86-6396-4ee0-9086-7b93ef8e4a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:46.967782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e3c86-6396-4ee0-9086-7b93ef8e4a68 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:46.968627 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e3c86-6396-4ee0-9086-7b93ef8e4a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:46.969356 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e3c86-6396-4ee0-9086-7b93ef8e4a68 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:46.971372 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e3c86-6396-4ee0-9086-7b93ef8e4a68 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:46.972472 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e3c86-6396-4ee0-9086-7b93ef8e4a68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:46.976214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e3c86-6396-4ee0-9086-7b93ef8e4a68 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:46.987439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 469/940] 10.222.0.27 () {56 vars in 1122 bytes} [Sat Jul 16 14:50:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:46.996805 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e3c86-6396-4ee0-9086-7b93ef8e4a68 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:47.142141 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d6079-1089-40cd-aaee-90521eec3e63 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:47.186092 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e3c86-6396-4ee0-9086-7b93ef8e4a68 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:47.206491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27069ea-e3ad-4f97-a502-828968e9d87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:47.207953 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27069ea-e3ad-4f97-a502-828968e9d87c None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:47.208677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27069ea-e3ad-4f97-a502-828968e9d87c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:47.208936 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27069ea-e3ad-4f97-a502-828968e9d87c None admin] PATH_INFO: `/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/a4f499707998403d8f833eb783599466/roles/e3c9141bd01741b59c8701275be9154c` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:47.213775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27069ea-e3ad-4f97-a502-828968e9d87c None admin] RBAC: Authorizing `identity:create_grant(project_id=2739f00eb01e46a9acf90ffa360d7d8c, user_id=a4f499707998403d8f833eb783599466, role_id=e3c9141bd01741b59c8701275be9154c)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:47.216562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27069ea-e3ad-4f97-a502-828968e9d87c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:47.244541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41785e-abc5-413b-ac18-42796b7b9208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:47.245451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41785e-abc5-413b-ac18-42796b7b9208 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:47.245980 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41785e-abc5-413b-ac18-42796b7b9208 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:47.246441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41785e-abc5-413b-ac18-42796b7b9208 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:47.247419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41785e-abc5-413b-ac18-42796b7b9208 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:47.247941 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41785e-abc5-413b-ac18-42796b7b9208 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:47.249594 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41785e-abc5-413b-ac18-42796b7b9208 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:47.258363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 470/941] 10.222.0.27 () {56 vars in 1122 bytes} [Sat Jul 16 14:50:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:47.264256 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41785e-abc5-413b-ac18-42796b7b9208 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:47.298348 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:47.327978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 472/942] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:50:46 2022] PUT /identity/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/a4f499707998403d8f833eb783599466/roles/e3c9141bd01741b59c8701275be9154c => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 14:50:47.409796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41785e-abc5-413b-ac18-42796b7b9208 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:47.486196 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6dc8a0-6214-4224-9769-98b02b4b1967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:47.487914 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6dc8a0-6214-4224-9769-98b02b4b1967 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:47.488711 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6dc8a0-6214-4224-9769-98b02b4b1967 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:47.489545 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6dc8a0-6214-4224-9769-98b02b4b1967 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:47.492465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6dc8a0-6214-4224-9769-98b02b4b1967 None admin] RBAC: Adding query filter params (interface=public, service_id=d954fd0d0e1041379a96476c9eebabeb, region_id=RegionOne) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:47.493461 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6dc8a0-6214-4224-9769-98b02b4b1967 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:47.496571 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6dc8a0-6214-4224-9769-98b02b4b1967 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:47.508234 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 471/943] 10.222.0.27 () {56 vars in 1251 bytes} [Sat Jul 16 14:50:47 2022] GET /identity/v3/endpoints?service_id=d954fd0d0e1041379a96476c9eebabeb&interface=public®ion_id=RegionOne => generated 194 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:47.516098 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27069ea-e3ad-4f97-a502-828968e9d87c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:47.626089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:47.699791 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27069ea-e3ad-4f97-a502-828968e9d87c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:47.767458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467e882-a420-4033-b605-b9a8addabcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:47.768433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467e882-a420-4033-b605-b9a8addabcca None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:47.769146 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467e882-a420-4033-b605-b9a8addabcca None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:47.769612 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467e882-a420-4033-b605-b9a8addabcca None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:47.770803 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467e882-a420-4033-b605-b9a8addabcca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:47.774740 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467e882-a420-4033-b605-b9a8addabcca None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:47.788834 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 473/944] 10.222.0.27 () {56 vars in 1087 bytes} [Sat Jul 16 14:50:47 2022] GET /identity/v3/services => generated 1497 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 14:50:48.166072 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46ce3c-9ed3-44bc-9e8d-f33237945a58 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:48.166572 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46ce3c-9ed3-44bc-9e8d-f33237945a58 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:48.167090 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46ce3c-9ed3-44bc-9e8d-f33237945a58 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:48.170126 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 472/945] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:48.179973 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241e3ee-013a-4859-a2ec-ac4550fc01d1 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:48.180470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241e3ee-013a-4859-a2ec-ac4550fc01d1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:48.180960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a241e3ee-013a-4859-a2ec-ac4550fc01d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:48.210818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a241e3ee-013a-4859-a2ec-ac4550fc01d1 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:48.213067 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a241e3ee-013a-4859-a2ec-ac4550fc01d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:48.345500 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 474/946] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:48.384655 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46ce3c-9ed3-44bc-9e8d-f33237945a58 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:48.433758 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3e8f3-351b-4f57-80c2-f1747fe1c88f None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:48.434175 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3e8f3-351b-4f57-80c2-f1747fe1c88f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:48.434671 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3e8f3-351b-4f57-80c2-f1747fe1c88f None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:48.438688 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 475/947] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:48.447716 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505590e2-68bd-4911-8888-61a7e77414a2 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:48.448144 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505590e2-68bd-4911-8888-61a7e77414a2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:48.448687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505590e2-68bd-4911-8888-61a7e77414a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:48.470568 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-505590e2-68bd-4911-8888-61a7e77414a2 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:48.473255 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-505590e2-68bd-4911-8888-61a7e77414a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:48.534822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:48.555161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:48.556442 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 476/948] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:48.571445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46ce3c-9ed3-44bc-9e8d-f33237945a58 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:48.575795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505590e2-68bd-4911-8888-61a7e77414a2 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:48.620934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d415ab0-485f-4a84-a4f0-d8a391407615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:48.622021 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d415ab0-485f-4a84-a4f0-d8a391407615 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:48.622501 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d415ab0-485f-4a84-a4f0-d8a391407615 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:48.623012 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d415ab0-485f-4a84-a4f0-d8a391407615 None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:48.623933 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d415ab0-485f-4a84-a4f0-d8a391407615 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:48.625591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d415ab0-485f-4a84-a4f0-d8a391407615 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:48.627395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: WARNING keystone.server.flask.common [None req-4d415ab0-485f-4a84-a4f0-d8a391407615 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 16 14:50:48.692947 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.sql.core [None req-4d415ab0-485f-4a84-a4f0-d8a391407615 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 16 14:50:48.692947 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 16 14:50:48.692947 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [parameters: {'id': '565c796aa8304f80ab380832836fac59', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 16 14:50:48.692947 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80257) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 16 14:50:48.701311 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505590e2-68bd-4911-8888-61a7e77414a2 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:48.716089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-4d415ab0-485f-4a84-a4f0-d8a391407615 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 565c796aa8304f80ab380832836fac59.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 565c796aa8304f80ab380832836fac59. Jul 16 14:50:48.716089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:48.716089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 16 14:50:48.716089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 16 14:50:48.716089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 16 14:50:48.716089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 16 14:50:48.716089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 16 14:50:48.716089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application result = self._query(query) Jul 16 14:50:48.716089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 16 14:50:48.716089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application conn.query(q) Jul 16 14:50:48.716089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 16 14:50:48.716089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 16 14:50:48.716089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 16 14:50:48.716982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application result.read() Jul 16 14:50:48.716982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 16 14:50:48.716982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 16 14:50:48.716982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 16 14:50:48.716982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 16 14:50:48.716982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 16 14:50:48.716982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 16 14:50:48.716982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 16 14:50:48.716982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 16 14:50:48.716982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 16 14:50:48.716982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:48.716982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 16 14:50:48.716982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:48.716982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:48.716982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 16 14:50:48.716982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 16 14:50:48.716982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 16 14:50:48.716982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 16 14:50:48.717951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 16 14:50:48.717951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application next(self.gen) Jul 16 14:50:48.717951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 16 14:50:48.717951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application yield resource Jul 16 14:50:48.717951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 16 14:50:48.717951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application next(self.gen) Jul 16 14:50:48.717951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 16 14:50:48.717951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self.session.rollback() Jul 16 14:50:48.717951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 14:50:48.717951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self.force_reraise() Jul 16 14:50:48.717951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 14:50:48.717951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise self.value Jul 16 14:50:48.717951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 16 14:50:48.717951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 16 14:50:48.717951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 16 14:50:48.717951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application session.commit() Jul 16 14:50:48.717951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 16 14:50:48.718851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 16 14:50:48.718851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 16 14:50:48.718851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._prepare_impl() Jul 16 14:50:48.718851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 16 14:50:48.718851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self.session.flush() Jul 16 14:50:48.718851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 16 14:50:48.718851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._flush(objects) Jul 16 14:50:48.718851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 16 14:50:48.718851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 16 14:50:48.718851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 16 14:50:48.718851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application compat.raise_( Jul 16 14:50:48.718851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 16 14:50:48.718851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception Jul 16 14:50:48.718851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 16 14:50:48.718851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application flush_context.execute() Jul 16 14:50:48.718851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 16 14:50:48.718851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rec.execute(self) Jul 16 14:50:48.719796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 16 14:50:48.719796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 16 14:50:48.719796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 16 14:50:48.719796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 16 14:50:48.719796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 16 14:50:48.719796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 16 14:50:48.719796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 16 14:50:48.719796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 16 14:50:48.719796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 16 14:50:48.719796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 16 14:50:48.719796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 16 14:50:48.719796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 16 14:50:48.719796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 16 14:50:48.719796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 16 14:50:48.719796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 16 14:50:48.720593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 16 14:50:48.720593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 16 14:50:48.720593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception Jul 16 14:50:48.720593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 16 14:50:48.720593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 16 14:50:48.720593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 16 14:50:48.720593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 16 14:50:48.720593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 16 14:50:48.720593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application result = self._query(query) Jul 16 14:50:48.720593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 16 14:50:48.720593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application conn.query(q) Jul 16 14:50:48.720593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 16 14:50:48.720593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 16 14:50:48.720593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 16 14:50:48.720593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application result.read() Jul 16 14:50:48.720593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 16 14:50:48.720593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 16 14:50:48.721471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 16 14:50:48.721471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 16 14:50:48.721471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 16 14:50:48.721471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 16 14:50:48.721471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 16 14:50:48.721471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 16 14:50:48.721471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 16 14:50:48.721471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 16 14:50:48.721471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [parameters: {'id': '565c796aa8304f80ab380832836fac59', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 16 14:50:48.721471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 16 14:50:48.721471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:48.721471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:48.721471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:48.721471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:48.721471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 16 14:50:48.724441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 16 14:50:48.724441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 16 14:50:48.724441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 16 14:50:48.724441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 16 14:50:48.724441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:48.724441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:48.724441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:48.724441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:48.724441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:48.724441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:48.724441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:48.724441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:48.724441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:48.724441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:48.724441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:48.724441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:48.724441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:48.727215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:48.727215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 16 14:50:48.727215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 16 14:50:48.727215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:48.727215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:48.727215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 16 14:50:48.727215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 16 14:50:48.727215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 565c796aa8304f80ab380832836fac59. Jul 16 14:50:48.727215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:48.728772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 473/949] 10.222.0.27 () {60 vars in 1140 bytes} [Sat Jul 16 14:50:48 2022] POST /identity/v3/projects => generated 262 bytes in 344 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:48.733655 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d415ab0-485f-4a84-a4f0-d8a391407615 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:48.767817 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17686e5-59ab-4a6b-b16c-1312b8ce01d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:48.770128 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17686e5-59ab-4a6b-b16c-1312b8ce01d4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:48.770652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17686e5-59ab-4a6b-b16c-1312b8ce01d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:48.771189 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17686e5-59ab-4a6b-b16c-1312b8ce01d4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:48.773195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17686e5-59ab-4a6b-b16c-1312b8ce01d4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:48.775238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17686e5-59ab-4a6b-b16c-1312b8ce01d4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:48.795691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-a17686e5-59ab-4a6b-b16c-1312b8ce01d4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 16 14:50:48.795691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:48.795691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:48.795691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:48.795691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:48.795691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:48.795691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:48.795691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:48.795691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 16 14:50:48.795691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:48.795691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:48.795691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:48.795691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:48.795691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:48.795691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:48.795691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:48.795691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:48.796960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:48.796960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:48.796960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:48.796960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:48.796960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:48.796960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:48.796960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:48.796960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:48.796960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:48.796960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:48.796960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:48.796960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:48.796960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:48.796960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:48.796960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:48.796960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:48.797889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:48.797889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:48.797889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:48.797889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:48.797889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:48.797889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:48.797889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:48.797889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:48.797889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:48.797889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:48.797889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:48.797889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:48.797889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 16 14:50:48.797889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:48.803876 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 477/950] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:48.806522 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17686e5-59ab-4a6b-b16c-1312b8ce01d4 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:48.884346 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:48.896372 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d415ab0-485f-4a84-a4f0-d8a391407615 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:48.965862 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17686e5-59ab-4a6b-b16c-1312b8ce01d4 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:48.970105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabdbcd-8638-4f4e-a189-c940a7c0301b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:48.972565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabdbcd-8638-4f4e-a189-c940a7c0301b None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:48.973222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabdbcd-8638-4f4e-a189-c940a7c0301b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:48.973707 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabdbcd-8638-4f4e-a189-c940a7c0301b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:48.985001 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aabdbcd-8638-4f4e-a189-c940a7c0301b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:48.988088 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aabdbcd-8638-4f4e-a189-c940a7c0301b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:49.008722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-3aabdbcd-8638-4f4e-a189-c940a7c0301b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:49.008722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:49.008722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:49.008722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:49.008722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:49.008722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:49.008722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:49.008722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:49.008722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:49.008722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:49.008722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:49.008722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:49.008722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:50:49.008722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:50:49.008722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:50:49.009712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:50:49.009712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:49.009712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:49.009712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:49.009712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:49.009712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:49.009712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:49.009712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:49.009712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:49.009712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:49.009712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:49.009712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:49.009712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:49.009712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:49.009712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:49.009712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:49.009712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:49.010695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:50:49.010695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:50:49.010695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:50:49.010695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:50:49.010695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:50:49.010695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:50:49.010695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:49.010695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:49.020188 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 474/951] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:49.024375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabdbcd-8638-4f4e-a189-c940a7c0301b None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:49.057699 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:49.059288 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bd676-d0b4-48f2-b5bb-6f7b2ee61156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:49.061613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bd676-d0b4-48f2-b5bb-6f7b2ee61156 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:49.062118 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bd676-d0b4-48f2-b5bb-6f7b2ee61156 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:49.062616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bd676-d0b4-48f2-b5bb-6f7b2ee61156 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:49.064097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021bd676-d0b4-48f2-b5bb-6f7b2ee61156 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:49.064670 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021bd676-d0b4-48f2-b5bb-6f7b2ee61156 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:49.067369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021bd676-d0b4-48f2-b5bb-6f7b2ee61156 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:49.083065 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 478/952] 10.222.0.27 () {56 vars in 1112 bytes} [Sat Jul 16 14:50:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:49.091099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bd676-d0b4-48f2-b5bb-6f7b2ee61156 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:49.233638 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabdbcd-8638-4f4e-a189-c940a7c0301b None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:49.312087 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c346603-0e0a-44ff-99b0-5be831ceb9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:49.313966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c346603-0e0a-44ff-99b0-5be831ceb9fa None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:49.315545 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c346603-0e0a-44ff-99b0-5be831ceb9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:49.316067 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c346603-0e0a-44ff-99b0-5be831ceb9fa None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:49.317598 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c346603-0e0a-44ff-99b0-5be831ceb9fa None admin] RBAC: Adding query filter params (name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:49.318143 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c346603-0e0a-44ff-99b0-5be831ceb9fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:49.319085 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bd676-d0b4-48f2-b5bb-6f7b2ee61156 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:49.321056 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c346603-0e0a-44ff-99b0-5be831ceb9fa None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:49.355215 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:49.372528 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 475/953] 10.222.0.27 () {56 vars in 1112 bytes} [Sat Jul 16 14:50:49 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:49.398993 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5035489-4aa1-4837-a17a-cc63a0b0afa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:49.400845 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5035489-4aa1-4837-a17a-cc63a0b0afa7 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:49.401713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5035489-4aa1-4837-a17a-cc63a0b0afa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:49.402505 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5035489-4aa1-4837-a17a-cc63a0b0afa7 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:49.404266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5035489-4aa1-4837-a17a-cc63a0b0afa7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:49.405100 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5035489-4aa1-4837-a17a-cc63a0b0afa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:49.408204 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5035489-4aa1-4837-a17a-cc63a0b0afa7 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:49.419349 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 479/954] 10.222.0.27 () {56 vars in 1112 bytes} [Sat Jul 16 14:50:49 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:49.942362 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53fb497-8278-4bb1-bfe4-a25ae97d5876 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:49.942874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53fb497-8278-4bb1-bfe4-a25ae97d5876 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:49.942979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53fb497-8278-4bb1-bfe4-a25ae97d5876 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:49.945225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 476/955] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:49.950899 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a6617-66a0-4dae-9d2f-e2d8c8049dd0 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:49.951199 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a6617-66a0-4dae-9d2f-e2d8c8049dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:49.951474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a6617-66a0-4dae-9d2f-e2d8c8049dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:49.973144 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-9a9a6617-66a0-4dae-9d2f-e2d8c8049dd0 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:49.974448 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9a9a6617-66a0-4dae-9d2f-e2d8c8049dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:50.082284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 480/956] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:50.113469 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53fb497-8278-4bb1-bfe4-a25ae97d5876 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:50.194514 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6ec3a-19f6-43a4-94d7-260ca9fe7bee None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:50.195187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6ec3a-19f6-43a4-94d7-260ca9fe7bee None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:50.195812 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6ec3a-19f6-43a4-94d7-260ca9fe7bee None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:50.197857 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 481/957] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:50.204116 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd93b6f-8a4a-4f89-bb1e-44eea1ead980 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:50.204790 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd93b6f-8a4a-4f89-bb1e-44eea1ead980 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:50.205333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd93b6f-8a4a-4f89-bb1e-44eea1ead980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:50.224798 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-bbd93b6f-8a4a-4f89-bb1e-44eea1ead980 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:50.227359 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bbd93b6f-8a4a-4f89-bb1e-44eea1ead980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:50.240323 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:50.320484 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53fb497-8278-4bb1-bfe4-a25ae97d5876 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:50.321244 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 482/958] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:50:50.340525 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd93b6f-8a4a-4f89-bb1e-44eea1ead980 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:50.391195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c67d1-c1f8-43bf-b112-2dcc6ed66f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:50.392604 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c67d1-c1f8-43bf-b112-2dcc6ed66f9a None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:50.393117 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c67d1-c1f8-43bf-b112-2dcc6ed66f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:50.393935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:50.394377 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c67d1-c1f8-43bf-b112-2dcc6ed66f9a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:50.395503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c67d1-c1f8-43bf-b112-2dcc6ed66f9a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:50.398127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c67d1-c1f8-43bf-b112-2dcc6ed66f9a None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:50.413794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-1d6c67d1-c1f8-43bf-b112-2dcc6ed66f9a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 14:50:50.413794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:50.413794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:50.413794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:50.413794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:50.413794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:50.413794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:50.413794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:50.413794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 14:50:50.413794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:50.413794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:50.413794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:50.413794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:50.413794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:50.413794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:50.413794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:50.413794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:50.414865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:50.414865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:50.414865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:50.414865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:50.414865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:50.414865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:50.414865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:50.414865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:50.414865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:50.414865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:50.414865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:50.414865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:50.414865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:50.414865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:50.414865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:50.414865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:50.415730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:50.415730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:50.415730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:50.415730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:50.415730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:50.415730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:50.415730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:50.415730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:50.415730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:50.415730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:50.415730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:50.415730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:50.415730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 14:50:50.415730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:50.418968 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 477/959] 10.222.0.27 () {56 vars in 1126 bytes} [Sat Jul 16 14:50:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:50.423160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c67d1-c1f8-43bf-b112-2dcc6ed66f9a None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:50.527395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd93b6f-8a4a-4f89-bb1e-44eea1ead980 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:50.559174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c67d1-c1f8-43bf-b112-2dcc6ed66f9a None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:50.599240 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:50.607601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8d543-31a0-4a16-9412-fea054961187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:50.609332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8d543-31a0-4a16-9412-fea054961187 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:50.610144 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8d543-31a0-4a16-9412-fea054961187 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:50.610755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8d543-31a0-4a16-9412-fea054961187 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:50.611745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8d543-31a0-4a16-9412-fea054961187 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:50.612232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8d543-31a0-4a16-9412-fea054961187 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:50.613864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8d543-31a0-4a16-9412-fea054961187 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:50.619930 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229377a0-f040-4058-8565-158ca0501fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:50.621432 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229377a0-f040-4058-8565-158ca0501fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:50.622077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229377a0-f040-4058-8565-158ca0501fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:50.622506 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229377a0-f040-4058-8565-158ca0501fe9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:50.622847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 478/960] 10.222.0.27 () {56 vars in 1122 bytes} [Sat Jul 16 14:50:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:50.628785 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8d543-31a0-4a16-9412-fea054961187 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:50.651213 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229377a0-f040-4058-8565-158ca0501fe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:50:50.660197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-229377a0-f040-4058-8565-158ca0501fe9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 14:50:50.660197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:50.660197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:50.660197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:50.660197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:50.660197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:50.660197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:50.660197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:50.660197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:50.660197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:50.660197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:50.660197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:50.660197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:50:50.660197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:50:50.660197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:50:50.660197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:50:50.661195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:50:50.661195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:50:50.661195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:50:50.661195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:50:50.661195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:50.661195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:50.661195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:50.661195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:50.661195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:50.661195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:50.661195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:50.661195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:50.661195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:50.661195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:50.661195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:50.661195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:50.662018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:50.662018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:50.662018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:50.662018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:50.662018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:50:50.662018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:50:50.662018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:50:50.662018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:50:50.662018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:50:50.662018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:50:50.662018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 14:50:50.662018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:50.670969 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 483/961] 10.222.0.27 () {56 vars in 1120 bytes} [Sat Jul 16 14:50:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:50.673387 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229377a0-f040-4058-8565-158ca0501fe9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:50.791275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:50.794491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229377a0-f040-4058-8565-158ca0501fe9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:50.835364 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8d543-31a0-4a16-9412-fea054961187 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:50.836056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e0469-0d2e-494a-812c-0d8bfd534e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:50.837361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e0469-0d2e-494a-812c-0d8bfd534e29 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:50.837852 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e0469-0d2e-494a-812c-0d8bfd534e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:50.838443 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e0469-0d2e-494a-812c-0d8bfd534e29 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:50.839473 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e0469-0d2e-494a-812c-0d8bfd534e29 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:50.840106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e0469-0d2e-494a-812c-0d8bfd534e29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:50.841918 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e0469-0d2e-494a-812c-0d8bfd534e29 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:50.858306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 484/962] 10.222.0.27 () {56 vars in 1115 bytes} [Sat Jul 16 14:50:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:50.859737 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e0469-0d2e-494a-812c-0d8bfd534e29 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:50.929660 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60802d6-4a46-4da5-aa73-b94f3c1f28a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:50.931089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60802d6-4a46-4da5-aa73-b94f3c1f28a4 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:50.931597 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60802d6-4a46-4da5-aa73-b94f3c1f28a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:50.932086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60802d6-4a46-4da5-aa73-b94f3c1f28a4 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:50.933571 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60802d6-4a46-4da5-aa73-b94f3c1f28a4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:50.934488 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60802d6-4a46-4da5-aa73-b94f3c1f28a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:50.936936 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60802d6-4a46-4da5-aa73-b94f3c1f28a4 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:50.948717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 479/963] 10.222.0.27 () {56 vars in 1122 bytes} [Sat Jul 16 14:50:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:50.956060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60802d6-4a46-4da5-aa73-b94f3c1f28a4 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:51.002925 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e0469-0d2e-494a-812c-0d8bfd534e29 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:51.047366 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:51.053809 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad06fc-413a-44f6-8424-8c76e76f5cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:51.054837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad06fc-413a-44f6-8424-8c76e76f5cfc None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:51.054837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad06fc-413a-44f6-8424-8c76e76f5cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:51.055189 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad06fc-413a-44f6-8424-8c76e76f5cfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:51.067893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad06fc-413a-44f6-8424-8c76e76f5cfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:51.069985 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad06fc-413a-44f6-8424-8c76e76f5cfc None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:51.090132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-14ad06fc-413a-44f6-8424-8c76e76f5cfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:51.090132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:51.090132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:51.090132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:51.090132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:51.090132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:51.090132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:51.090132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:51.090132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:51.090132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:51.090132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:51.090132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:51.090132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:51.090132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:51.090132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:51.091137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:51.091137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:51.091137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:51.091137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:51.091137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:51.091137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:51.091137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:51.091137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:51.091137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:51.091137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:51.091137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:51.091137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:51.091137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:51.091137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:51.091137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:51.091137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:51.091137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:51.092034 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:51.092034 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:51.092034 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:51.092034 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:51.096301 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 485/964] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:51.097299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad06fc-413a-44f6-8424-8c76e76f5cfc None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:51.116965 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:51.137312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60802d6-4a46-4da5-aa73-b94f3c1f28a4 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:51.229332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a5768-95b0-4981-b440-d6defec4fe08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:51.231238 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a5768-95b0-4981-b440-d6defec4fe08 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:51.231948 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a5768-95b0-4981-b440-d6defec4fe08 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:51.232719 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a5768-95b0-4981-b440-d6defec4fe08 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:51.235355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a5768-95b0-4981-b440-d6defec4fe08 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:51.237999 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a5768-95b0-4981-b440-d6defec4fe08 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:51.258511 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad06fc-413a-44f6-8424-8c76e76f5cfc None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:51.338002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff52819-ff41-4908-8d1c-4889b580b832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:51.339226 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff52819-ff41-4908-8d1c-4889b580b832 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:51.339770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff52819-ff41-4908-8d1c-4889b580b832 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:51.340259 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff52819-ff41-4908-8d1c-4889b580b832 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:51.341658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff52819-ff41-4908-8d1c-4889b580b832 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:51.342206 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff52819-ff41-4908-8d1c-4889b580b832 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:51.344974 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff52819-ff41-4908-8d1c-4889b580b832 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:51.351751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 480/965] 10.222.0.27 () {60 vars in 1144 bytes} [Sat Jul 16 14:50:50 2022] POST /identity/v3/endpoints => generated 349 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:50:51.375264 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 486/966] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:51.384584 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a5768-95b0-4981-b440-d6defec4fe08 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:51.541600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a5768-95b0-4981-b440-d6defec4fe08 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:51.557811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:51.583096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3504e-3b92-4b49-a8e7-945d64992f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:51.584033 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3504e-3b92-4b49-a8e7-945d64992f35 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:51.584313 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3504e-3b92-4b49-a8e7-945d64992f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:51.585233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3504e-3b92-4b49-a8e7-945d64992f35 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:51.594839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3504e-3b92-4b49-a8e7-945d64992f35 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:51.596329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3504e-3b92-4b49-a8e7-945d64992f35 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:51.615700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-c2b3504e-3b92-4b49-a8e7-945d64992f35 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 14:50:51.615700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:51.615700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:51.615700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:51.615700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:51.615700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:51.615700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:51.615700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:51.615700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:51.615700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:51.615700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:51.615700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:51.615700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:50:51.615700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:50:51.615700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:50:51.615700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:50:51.617995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:51.617995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:51.617995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:50:51.617995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:51.617995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:50:51.617995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:51.617995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:51.617995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:51.617995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:51.617995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:51.617995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:51.617995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:51.617995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:51.617995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:51.617995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:51.617995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:51.617995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:51.617995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:51.620592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:51.620592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:51.620592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:50:51.620592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:50:51.620592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:50:51.620592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:50:51.620592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:50:51.620592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:50:51.620592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 16 14:50:51.620592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:51.620592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 481/967] 10.222.0.27 () {56 vars in 1117 bytes} [Sat Jul 16 14:50:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:51.623714 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff52819-ff41-4908-8d1c-4889b580b832 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:51.683757 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67f68d-e2a8-42fc-9885-4777db9b2b22 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:51.684186 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67f68d-e2a8-42fc-9885-4777db9b2b22 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:51.684425 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67f68d-e2a8-42fc-9885-4777db9b2b22 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:51.686125 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 482/968] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:50:51.696504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa6417-ab1b-4c5a-b777-70cea4466506 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:51.697474 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa6417-ab1b-4c5a-b777-70cea4466506 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:51.698260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa6417-ab1b-4c5a-b777-70cea4466506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:51.729857 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-8cfa6417-ab1b-4c5a-b777-70cea4466506 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:51.731341 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:51.732932 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-8cfa6417-ab1b-4c5a-b777-70cea4466506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:51.753381 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff52819-ff41-4908-8d1c-4889b580b832 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:51.799179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d2b2e-3c5d-4e17-80f6-5b5e33994401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:51.800098 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d2b2e-3c5d-4e17-80f6-5b5e33994401 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:51.800534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d2b2e-3c5d-4e17-80f6-5b5e33994401 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:51.800977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d2b2e-3c5d-4e17-80f6-5b5e33994401 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:51.802288 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d2b2e-3c5d-4e17-80f6-5b5e33994401 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:51.802935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d2b2e-3c5d-4e17-80f6-5b5e33994401 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:51.804689 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d2b2e-3c5d-4e17-80f6-5b5e33994401 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:51.835324 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 487/969] 10.222.0.27 () {56 vars in 1149 bytes} [Sat Jul 16 14:50:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:51.835324 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f8c3e-ab49-4d9e-a39b-7ce9a7c8ad54 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:51.835324 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f8c3e-ab49-4d9e-a39b-7ce9a7c8ad54 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:51.835324 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f8c3e-ab49-4d9e-a39b-7ce9a7c8ad54 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:51.835324 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 488/970] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:51.838856 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f8c3e-ab49-4d9e-a39b-7ce9a7c8ad54 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:51.847017 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 483/971] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:51.847017 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96051d6-3b14-4f9f-a219-15033ad5007c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:51.847017 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96051d6-3b14-4f9f-a219-15033ad5007c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:51.847017 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96051d6-3b14-4f9f-a219-15033ad5007c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:51.874594 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-e96051d6-3b14-4f9f-a219-15033ad5007c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:51.882284 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e96051d6-3b14-4f9f-a219-15033ad5007c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:51.974110 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 484/972] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:50:51.975155 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96051d6-3b14-4f9f-a219-15033ad5007c None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:51.988941 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f8c3e-ab49-4d9e-a39b-7ce9a7c8ad54 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:52.020122 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:52.028763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:52.034162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564495a-a718-4454-b793-9c62e3a66f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:52.034819 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564495a-a718-4454-b793-9c62e3a66f02 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:52.035118 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564495a-a718-4454-b793-9c62e3a66f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:52.035492 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564495a-a718-4454-b793-9c62e3a66f02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:52.103341 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96051d6-3b14-4f9f-a219-15033ad5007c None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:52.125236 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564495a-a718-4454-b793-9c62e3a66f02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:52.126929 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564495a-a718-4454-b793-9c62e3a66f02 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:52.158272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09d681-8c7d-4e84-9f39-3876af3954de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:52.159435 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09d681-8c7d-4e84-9f39-3876af3954de None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:52.159914 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09d681-8c7d-4e84-9f39-3876af3954de None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:52.160389 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09d681-8c7d-4e84-9f39-3876af3954de None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:52.161667 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a09d681-8c7d-4e84-9f39-3876af3954de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:52.164911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a09d681-8c7d-4e84-9f39-3876af3954de None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:52.214808 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.sql.core [None req-2a09d681-8c7d-4e84-9f39-3876af3954de None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 16 14:50:52.214808 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 16 14:50:52.214808 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [parameters: {'user_id': 'ce7f03ca4a974d5e87f3d100dc93e3c1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 16 14:50:52.214808 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80257) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 16 14:50:52.224609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-2a09d681-8c7d-4e84-9f39-3876af3954de None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 16 14:50:52.224609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:52.224609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 16 14:50:52.224609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 16 14:50:52.224609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 16 14:50:52.224609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 16 14:50:52.224609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 16 14:50:52.224609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application result = self._query(query) Jul 16 14:50:52.224609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 16 14:50:52.224609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application conn.query(q) Jul 16 14:50:52.224609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 16 14:50:52.224609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 16 14:50:52.224609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 16 14:50:52.224609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application result.read() Jul 16 14:50:52.224609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 16 14:50:52.227174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 16 14:50:52.227174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 16 14:50:52.227174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 16 14:50:52.227174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 16 14:50:52.227174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 16 14:50:52.227174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 16 14:50:52.227174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 16 14:50:52.227174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 16 14:50:52.227174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:52.227174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 16 14:50:52.227174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:52.227174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:52.227174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 16 14:50:52.227174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 16 14:50:52.227174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 16 14:50:52.227174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 16 14:50:52.227174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 16 14:50:52.227174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application next(self.gen) Jul 16 14:50:52.229181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 16 14:50:52.229181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application yield resource Jul 16 14:50:52.229181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 16 14:50:52.229181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application next(self.gen) Jul 16 14:50:52.229181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 16 14:50:52.229181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self.session.rollback() Jul 16 14:50:52.229181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 14:50:52.229181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self.force_reraise() Jul 16 14:50:52.229181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 14:50:52.229181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise self.value Jul 16 14:50:52.229181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 16 14:50:52.229181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 16 14:50:52.229181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 16 14:50:52.229181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application session.commit() Jul 16 14:50:52.229181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 16 14:50:52.229181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 16 14:50:52.229181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 16 14:50:52.231402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._prepare_impl() Jul 16 14:50:52.231402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 16 14:50:52.231402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self.session.flush() Jul 16 14:50:52.231402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 16 14:50:52.231402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._flush(objects) Jul 16 14:50:52.231402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 16 14:50:52.231402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 16 14:50:52.231402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 16 14:50:52.231402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application compat.raise_( Jul 16 14:50:52.231402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 16 14:50:52.231402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception Jul 16 14:50:52.231402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 16 14:50:52.231402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application flush_context.execute() Jul 16 14:50:52.231402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 16 14:50:52.231402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rec.execute(self) Jul 16 14:50:52.231402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 16 14:50:52.231402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 16 14:50:52.232455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 16 14:50:52.232455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 16 14:50:52.232455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 16 14:50:52.232455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 16 14:50:52.232455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 16 14:50:52.232455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 16 14:50:52.232455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 16 14:50:52.232455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 16 14:50:52.232455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 16 14:50:52.232455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 16 14:50:52.232455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 16 14:50:52.232455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 16 14:50:52.232455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 16 14:50:52.232455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 16 14:50:52.232455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 16 14:50:52.232455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception Jul 16 14:50:52.234297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 16 14:50:52.234297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 16 14:50:52.234297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 16 14:50:52.234297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 16 14:50:52.234297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 16 14:50:52.234297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application result = self._query(query) Jul 16 14:50:52.234297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 16 14:50:52.234297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application conn.query(q) Jul 16 14:50:52.234297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 16 14:50:52.234297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 16 14:50:52.234297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 16 14:50:52.234297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application result.read() Jul 16 14:50:52.234297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 16 14:50:52.234297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 16 14:50:52.234297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 16 14:50:52.234297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 16 14:50:52.240136 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 16 14:50:52.240136 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 16 14:50:52.240136 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 16 14:50:52.240136 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 16 14:50:52.240136 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 16 14:50:52.240136 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 16 14:50:52.240136 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ce7f03ca4a974d5e87f3d100dc93e3c1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 16 14:50:52.240136 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 16 14:50:52.240136 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:52.240136 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:52.240136 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:52.240136 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:52.240136 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:52.240136 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:52.240136 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:52.242324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:52.242324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:52.242324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:52.242324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:52.242324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:52.242324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:52.242324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:52.242324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 16 14:50:52.242324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 16 14:50:52.242324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:52.242324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:52.242324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:50:52.242324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:52.242324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:50:52.242324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:52.242324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 16 14:50:52.242324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 16 14:50:52.243832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 16 14:50:52.243832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 16 14:50:52.243832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 16 14:50:52.243832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 16 14:50:52.243832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 16 14:50:52.243832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:52.243832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 485/973] 10.222.0.27 () {60 vars in 1131 bytes} [Sat Jul 16 14:50:51 2022] POST /identity/v3/users => generated 160 bytes in 254 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:52.243832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09d681-8c7d-4e84-9f39-3876af3954de None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:52.253462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 489/974] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:50:51 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 16 14:50:52.254333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564495a-a718-4454-b793-9c62e3a66f02 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:52.301394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:52.385324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:52.395606 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09d681-8c7d-4e84-9f39-3876af3954de None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:52.408711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564495a-a718-4454-b793-9c62e3a66f02 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:52.448881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787792d-6064-4a7d-9b30-793487596e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:52.449564 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787792d-6064-4a7d-9b30-793487596e54 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:52.449836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787792d-6064-4a7d-9b30-793487596e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:52.450110 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787792d-6064-4a7d-9b30-793487596e54 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:52.451054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787792d-6064-4a7d-9b30-793487596e54 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:52.452602 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787792d-6064-4a7d-9b30-793487596e54 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:52.468029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7165fc-ae7c-48c4-afc7-1ce0a8a4f202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:52.469342 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7165fc-ae7c-48c4-afc7-1ce0a8a4f202 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:52.469816 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7165fc-ae7c-48c4-afc7-1ce0a8a4f202 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:52.470263 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7165fc-ae7c-48c4-afc7-1ce0a8a4f202 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:52.482898 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7165fc-ae7c-48c4-afc7-1ce0a8a4f202 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:52.484367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7165fc-ae7c-48c4-afc7-1ce0a8a4f202 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:52.499705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-1e7165fc-ae7c-48c4-afc7-1ce0a8a4f202 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 16 14:50:52.499705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:52.499705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:52.499705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:52.499705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:52.499705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:52.499705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:52.499705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:52.499705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:52.499705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:52.499705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:52.499705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:52.499705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:50:52.499705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:50:52.499705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:50:52.499705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:50:52.507646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:52.507646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:52.507646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:50:52.507646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:52.507646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:50:52.507646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:50:52.507646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:52.507646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:52.507646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:52.507646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:52.507646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:52.507646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:52.507646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:52.507646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:52.507646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:52.507646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:52.507646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:52.507646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:52.508838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:52.508838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:52.508838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:50:52.508838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:50:52.508838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:50:52.508838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:50:52.508838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:50:52.508838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:50:52.508838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 16 14:50:52.508838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:52.508838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 490/975] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:50:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:52.508838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7165fc-ae7c-48c4-afc7-1ce0a8a4f202 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:52.525667 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 486/976] 10.222.0.27 () {60 vars in 1141 bytes} [Sat Jul 16 14:50:52 2022] POST /identity/v3/services => generated 242 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:50:52.526430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787792d-6064-4a7d-9b30-793487596e54 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:52.625833 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:52.643717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7165fc-ae7c-48c4-afc7-1ce0a8a4f202 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:52.726942 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa6e87-2310-4d16-8529-c1b2ac167880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:52.728519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa6e87-2310-4d16-8529-c1b2ac167880 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:52.729058 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa6e87-2310-4d16-8529-c1b2ac167880 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:52.729593 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa6e87-2310-4d16-8529-c1b2ac167880 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:52.730038 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787792d-6064-4a7d-9b30-793487596e54 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:52.749907 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa6e87-2310-4d16-8529-c1b2ac167880 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:52.752638 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa6e87-2310-4d16-8529-c1b2ac167880 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:52.792003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-4afa6e87-2310-4d16-8529-c1b2ac167880 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:52.792003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:52.792003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:52.792003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:52.792003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:52.792003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:52.792003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:52.792003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:52.792003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:52.792003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:52.792003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:52.792003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:52.792003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:50:52.792003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:50:52.792003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:50:52.792997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:50:52.792997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:52.792997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:52.792997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:52.792997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:52.792997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:52.792997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:52.792997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:52.792997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:52.792997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:52.792997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:52.792997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:52.792997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:52.792997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:52.792997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:52.792997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:52.792997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:52.794437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:50:52.794437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:50:52.794437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:50:52.794437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:52.794855 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 491/977] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:52.801782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78aa45a-a1c1-420b-b91f-68b8593ca42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:52.803283 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78aa45a-a1c1-420b-b91f-68b8593ca42f None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:52.803721 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78aa45a-a1c1-420b-b91f-68b8593ca42f None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:52.803854 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa6e87-2310-4d16-8529-c1b2ac167880 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:52.804170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78aa45a-a1c1-420b-b91f-68b8593ca42f None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:52.805625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78aa45a-a1c1-420b-b91f-68b8593ca42f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:52.806123 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78aa45a-a1c1-420b-b91f-68b8593ca42f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:52.808644 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78aa45a-a1c1-420b-b91f-68b8593ca42f None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:52.880721 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 487/978] 10.222.0.27 () {56 vars in 1101 bytes} [Sat Jul 16 14:50:52 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:52.955258 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa6e87-2310-4d16-8529-c1b2ac167880 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:53.026408 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a7e3d8-3ae5-4891-956e-f4c4616cc606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:53.027584 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7e3d8-3ae5-4891-956e-f4c4616cc606 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:53.028076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7e3d8-3ae5-4891-956e-f4c4616cc606 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:53.028566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a7e3d8-3ae5-4891-956e-f4c4616cc606 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:53.029950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a7e3d8-3ae5-4891-956e-f4c4616cc606 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:53.030487 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a7e3d8-3ae5-4891-956e-f4c4616cc606 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:53.033208 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a7e3d8-3ae5-4891-956e-f4c4616cc606 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:53.051449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:53.059189 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 492/979] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:53.065889 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78aa45a-a1c1-420b-b91f-68b8593ca42f None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:53.095237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:53.243834 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78aa45a-a1c1-420b-b91f-68b8593ca42f None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:53.305356 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b04520-1b6e-4184-95d2-8d04464bd6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:53.307195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b04520-1b6e-4184-95d2-8d04464bd6f0 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:53.307877 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b04520-1b6e-4184-95d2-8d04464bd6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:53.308548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b04520-1b6e-4184-95d2-8d04464bd6f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:53.318334 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b04520-1b6e-4184-95d2-8d04464bd6f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:53.321288 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b04520-1b6e-4184-95d2-8d04464bd6f0 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:53.339247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-70b04520-1b6e-4184-95d2-8d04464bd6f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:53.339247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:53.339247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:53.339247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:53.339247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:53.339247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:53.339247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:53.339247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:53.339247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:53.339247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:53.339247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:53.339247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:53.339247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:50:53.339247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:50:53.339247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:50:53.340278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:50:53.340278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:53.340278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:53.340278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:53.340278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:53.340278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:53.340278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:53.340278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:53.340278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:53.340278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:53.340278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:53.340278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:53.340278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:53.340278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:53.340278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:53.340278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:53.340278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:53.341257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:50:53.341257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:50:53.341257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:50:53.341257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:50:53.341257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:50:53.341257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:50:53.341257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:50:53.341257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:53.342774 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 488/980] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:53.353604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a7e3d8-3ae5-4891-956e-f4c4616cc606 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:53.509364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a7e3d8-3ae5-4891-956e-f4c4616cc606 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:53.569091 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd36a8-37e2-4cc2-ab06-e4dd7f9fb042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:53.569831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd36a8-37e2-4cc2-ab06-e4dd7f9fb042 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:53.570315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd36a8-37e2-4cc2-ab06-e4dd7f9fb042 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:53.570595 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd36a8-37e2-4cc2-ab06-e4dd7f9fb042 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:53.571450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd36a8-37e2-4cc2-ab06-e4dd7f9fb042 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:53.571756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd36a8-37e2-4cc2-ab06-e4dd7f9fb042 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:53.573224 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd36a8-37e2-4cc2-ab06-e4dd7f9fb042 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:53.591293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 493/981] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:50:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:53.595666 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b04520-1b6e-4184-95d2-8d04464bd6f0 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:53.686579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:53.721613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b04520-1b6e-4184-95d2-8d04464bd6f0 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:53.780632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6185e2d-23fb-4b68-ab38-aaf5982deb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:53.781836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6185e2d-23fb-4b68-ab38-aaf5982deb6e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:53.782349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6185e2d-23fb-4b68-ab38-aaf5982deb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:53.782999 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6185e2d-23fb-4b68-ab38-aaf5982deb6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:53.784490 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6185e2d-23fb-4b68-ab38-aaf5982deb6e None admin] RBAC: Adding query filter params (role.id=e3c9141bd01741b59c8701275be9154c, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=a4f499707998403d8f833eb783599466) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:53.785171 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6185e2d-23fb-4b68-ab38-aaf5982deb6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:53.788689 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6185e2d-23fb-4b68-ab38-aaf5982deb6e None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:53.806927 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 489/982] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:50:53 2022] GET /identity/v3/role_assignments?user.id=a4f499707998403d8f833eb783599466&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=e3c9141bd01741b59c8701275be9154c => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:53.852380 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91699d-4d0c-4c7f-8693-039be686eb8d None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:53.852936 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91699d-4d0c-4c7f-8693-039be686eb8d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:53.853157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91699d-4d0c-4c7f-8693-039be686eb8d None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:53.854761 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 494/983] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:53.861584 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b922bbf-21ea-4103-8c7c-ab5a03e58943 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:53.861854 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b922bbf-21ea-4103-8c7c-ab5a03e58943 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:53.862177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b922bbf-21ea-4103-8c7c-ab5a03e58943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:53.885453 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-6b922bbf-21ea-4103-8c7c-ab5a03e58943 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:53.887314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-6b922bbf-21ea-4103-8c7c-ab5a03e58943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:53.980801 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 490/984] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:50:54.006950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91699d-4d0c-4c7f-8693-039be686eb8d None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:54.098813 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:54.179865 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91699d-4d0c-4c7f-8693-039be686eb8d None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:54.241437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67502671-d5f7-4280-ab89-e11d05fee4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:54.242870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67502671-d5f7-4280-ab89-e11d05fee4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:54.243313 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67502671-d5f7-4280-ab89-e11d05fee4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:54.243807 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67502671-d5f7-4280-ab89-e11d05fee4f2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:54.244983 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67502671-d5f7-4280-ab89-e11d05fee4f2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:54.247125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67502671-d5f7-4280-ab89-e11d05fee4f2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:54.265367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-67502671-d5f7-4280-ab89-e11d05fee4f2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 14:50:54.265367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:54.265367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:54.265367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:54.265367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:54.265367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:54.265367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:54.265367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:54.265367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 14:50:54.265367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:54.265367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:54.265367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:54.265367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:54.265367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:54.265367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:54.265367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:54.265367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:54.267704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:54.267704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:54.267704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:54.267704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:54.267704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:54.267704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:54.267704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:54.267704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:54.267704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:54.267704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:54.267704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:54.267704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:54.267704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:54.267704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:54.267704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:54.267704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:54.269694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:54.269694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:54.269694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:54.269694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:54.269694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:54.269694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:54.269694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:54.269694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:54.269694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:54.269694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:54.269694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:54.269694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:54.269694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 14:50:54.269694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:54.274456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 495/985] 10.222.0.27 () {56 vars in 1126 bytes} [Sat Jul 16 14:50:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:54.275572 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b922bbf-21ea-4103-8c7c-ab5a03e58943 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:54.409143 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:54.453912 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b922bbf-21ea-4103-8c7c-ab5a03e58943 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:54.507871 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8974dbf-0c7d-4c46-9bba-64135ba4f319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:54.508724 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8974dbf-0c7d-4c46-9bba-64135ba4f319 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:54.509002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8974dbf-0c7d-4c46-9bba-64135ba4f319 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:54.509281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8974dbf-0c7d-4c46-9bba-64135ba4f319 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:54.510017 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8974dbf-0c7d-4c46-9bba-64135ba4f319 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:54.510316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8974dbf-0c7d-4c46-9bba-64135ba4f319 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:54.511742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8974dbf-0c7d-4c46-9bba-64135ba4f319 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:54.517744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 491/986] 10.222.0.27 () {56 vars in 1122 bytes} [Sat Jul 16 14:50:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:54.525218 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67502671-d5f7-4280-ab89-e11d05fee4f2 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:54.653888 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67502671-d5f7-4280-ab89-e11d05fee4f2 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:54.681714 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:54.694768 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a431f-bc27-48d5-a430-7b5c4b25956b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:54.695666 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a431f-bc27-48d5-a430-7b5c4b25956b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:54.696047 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a431f-bc27-48d5-a430-7b5c4b25956b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:54.696422 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a431f-bc27-48d5-a430-7b5c4b25956b None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:54.697245 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a431f-bc27-48d5-a430-7b5c4b25956b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:54.697651 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a431f-bc27-48d5-a430-7b5c4b25956b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:54.699171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a431f-bc27-48d5-a430-7b5c4b25956b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:54.710829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 496/987] 10.222.0.27 () {56 vars in 1122 bytes} [Sat Jul 16 14:50:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:54.712630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8974dbf-0c7d-4c46-9bba-64135ba4f319 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:54.830860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8974dbf-0c7d-4c46-9bba-64135ba4f319 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:54.863926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:54.871235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea494725-8a5a-41de-8abf-9e7237955a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:54.872009 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea494725-8a5a-41de-8abf-9e7237955a33 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:54.872358 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea494725-8a5a-41de-8abf-9e7237955a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:54.872707 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea494725-8a5a-41de-8abf-9e7237955a33 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:54.873591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea494725-8a5a-41de-8abf-9e7237955a33 None admin] RBAC: Adding query filter params (interface=admin, service_id=d954fd0d0e1041379a96476c9eebabeb, region_id=RegionOne) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:54.873972 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea494725-8a5a-41de-8abf-9e7237955a33 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:54.875451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea494725-8a5a-41de-8abf-9e7237955a33 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:54.883290 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 492/988] 10.222.0.27 () {56 vars in 1249 bytes} [Sat Jul 16 14:50:54 2022] GET /identity/v3/endpoints?service_id=d954fd0d0e1041379a96476c9eebabeb&interface=admin®ion_id=RegionOne => generated 193 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:54.889132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a431f-bc27-48d5-a430-7b5c4b25956b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:55.010480 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a431f-bc27-48d5-a430-7b5c4b25956b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:55.025343 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c37e9-8472-4447-b5ed-4324f77a87f9 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:55.025647 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c37e9-8472-4447-b5ed-4324f77a87f9 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:55.025996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c37e9-8472-4447-b5ed-4324f77a87f9 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:55.027581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 493/989] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:50:55.033777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16d0af-c1b3-4725-9429-c9a229728df8 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:55.034063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16d0af-c1b3-4725-9429-c9a229728df8 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:55.034349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16d0af-c1b3-4725-9429-c9a229728df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:55.053944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-6e16d0af-c1b3-4725-9429-c9a229728df8 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:55.054724 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87d8a6-ecab-4c77-b2ed-72374cebd548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:55.054932 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87d8a6-ecab-4c77-b2ed-72374cebd548 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:55.055206 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87d8a6-ecab-4c77-b2ed-72374cebd548 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:55.055573 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87d8a6-ecab-4c77-b2ed-72374cebd548 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:55.055804 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-6e16d0af-c1b3-4725-9429-c9a229728df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:55.056180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87d8a6-ecab-4c77-b2ed-72374cebd548 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:55.057556 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb87d8a6-ecab-4c77-b2ed-72374cebd548 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:55.074938 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 497/990] 10.222.0.27 () {56 vars in 1087 bytes} [Sat Jul 16 14:50:54 2022] GET /identity/v3/services => generated 1727 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 14:50:55.152300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 494/991] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:50:55.176039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87d8a6-ecab-4c77-b2ed-72374cebd548 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:55.243418 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:55.299892 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87d8a6-ecab-4c77-b2ed-72374cebd548 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:55.339476 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083cd65-0dec-4ee7-9042-806bb130ca4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:55.340365 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083cd65-0dec-4ee7-9042-806bb130ca4d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:55.340647 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083cd65-0dec-4ee7-9042-806bb130ca4d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:55.340921 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083cd65-0dec-4ee7-9042-806bb130ca4d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:55.341676 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083cd65-0dec-4ee7-9042-806bb130ca4d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:55.343111 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083cd65-0dec-4ee7-9042-806bb130ca4d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:55.356458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-8083cd65-0dec-4ee7-9042-806bb130ca4d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 16 14:50:55.356458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:55.356458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:55.356458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:55.356458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:55.356458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:55.356458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:55.356458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:55.356458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 16 14:50:55.356458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:55.356458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:55.356458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:55.356458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:55.356458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:55.356458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:55.356458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:55.356458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:55.357531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:55.357531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:55.357531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:55.357531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:55.357531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:55.357531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:55.357531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:55.357531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:55.357531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:55.357531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:55.357531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:55.357531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:55.357531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:55.357531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:55.357531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:55.357531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:55.358383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:55.358383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:55.358383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:55.358383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:55.358383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:55.358383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:55.358383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:55.358383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:55.358383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:55.358383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:55.358383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:55.358383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:55.358383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 16 14:50:55.358383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:55.358383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 498/992] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:55.365591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16d0af-c1b3-4725-9429-c9a229728df8 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:55.473821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16d0af-c1b3-4725-9429-c9a229728df8 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:55.499154 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:55.511328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5fe3ac-77b3-4907-a11d-5d0da1ad8e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:55.512007 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fe3ac-77b3-4907-a11d-5d0da1ad8e7e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:55.513266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fe3ac-77b3-4907-a11d-5d0da1ad8e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:55.513266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5fe3ac-77b3-4907-a11d-5d0da1ad8e7e None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:55.513442 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5fe3ac-77b3-4907-a11d-5d0da1ad8e7e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:55.513694 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5fe3ac-77b3-4907-a11d-5d0da1ad8e7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:55.515246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5fe3ac-77b3-4907-a11d-5d0da1ad8e7e None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:55.526547 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 495/993] 10.222.0.27 () {56 vars in 1112 bytes} [Sat Jul 16 14:50:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:55.527976 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083cd65-0dec-4ee7-9042-806bb130ca4d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:55.641382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083cd65-0dec-4ee7-9042-806bb130ca4d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:55.659203 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:55.679810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecac90f-6e6c-4332-a09b-e875f2930996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:55.680489 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecac90f-6e6c-4332-a09b-e875f2930996 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:55.680772 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecac90f-6e6c-4332-a09b-e875f2930996 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:55.681108 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecac90f-6e6c-4332-a09b-e875f2930996 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:55.681958 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecac90f-6e6c-4332-a09b-e875f2930996 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:55.682312 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecac90f-6e6c-4332-a09b-e875f2930996 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:55.683749 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecac90f-6e6c-4332-a09b-e875f2930996 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:55.689431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 499/994] 10.222.0.27 () {56 vars in 1112 bytes} [Sat Jul 16 14:50:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:55.695943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5fe3ac-77b3-4907-a11d-5d0da1ad8e7e None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:55.799833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5fe3ac-77b3-4907-a11d-5d0da1ad8e7e None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:55.835978 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214a0de-c607-4b57-a570-718048b6ad1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:55.836992 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214a0de-c607-4b57-a570-718048b6ad1e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:55.837276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214a0de-c607-4b57-a570-718048b6ad1e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:55.837553 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214a0de-c607-4b57-a570-718048b6ad1e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:55.838354 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214a0de-c607-4b57-a570-718048b6ad1e None admin] RBAC: Adding query filter params (interface=public, service_id=d9b95159992d457ba5c7795cc71662ee, region_id=RegionOne) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:55.838669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214a0de-c607-4b57-a570-718048b6ad1e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:55.840081 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214a0de-c607-4b57-a570-718048b6ad1e None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:55.851054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 496/995] 10.222.0.27 () {56 vars in 1251 bytes} [Sat Jul 16 14:50:55 2022] GET /identity/v3/endpoints?service_id=d9b95159992d457ba5c7795cc71662ee&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:55.851989 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecac90f-6e6c-4332-a09b-e875f2930996 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:55.962978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecac90f-6e6c-4332-a09b-e875f2930996 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:55.998214 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e77b54-f8c4-46cd-8cee-57792e46d507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:55.998878 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e77b54-f8c4-46cd-8cee-57792e46d507 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:55.999185 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e77b54-f8c4-46cd-8cee-57792e46d507 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:55.999461 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e77b54-f8c4-46cd-8cee-57792e46d507 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:56.000145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e77b54-f8c4-46cd-8cee-57792e46d507 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:56.001505 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e77b54-f8c4-46cd-8cee-57792e46d507 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:56.008611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 500/996] 10.222.0.27 () {56 vars in 1087 bytes} [Sat Jul 16 14:50:55 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 14:50:56.731538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f2c80-31a5-4b07-b918-c573ce0abbcc None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:56.732060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f2c80-31a5-4b07-b918-c573ce0abbcc None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:56.732060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f2c80-31a5-4b07-b918-c573ce0abbcc None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:56.733603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 497/997] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:56.742649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e89b8-74de-41cb-b436-63999d2f6ecb None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:56.743213 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e89b8-74de-41cb-b436-63999d2f6ecb None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:56.743715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e89b8-74de-41cb-b436-63999d2f6ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:56.772018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-621e89b8-74de-41cb-b436-63999d2f6ecb None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:56.774744 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-621e89b8-74de-41cb-b436-63999d2f6ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:56.853192 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:56.914553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 501/998] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:50:56.940790 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f2c80-31a5-4b07-b918-c573ce0abbcc None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:57.001792 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:57.090213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f2c80-31a5-4b07-b918-c573ce0abbcc None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:57.156907 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496e2c6-72f0-4568-b2c1-c58e7511ec39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:57.158299 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496e2c6-72f0-4568-b2c1-c58e7511ec39 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:57.158721 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496e2c6-72f0-4568-b2c1-c58e7511ec39 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:57.159149 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496e2c6-72f0-4568-b2c1-c58e7511ec39 None admin] PATH_INFO: `/v3/services/image` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:57.160414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496e2c6-72f0-4568-b2c1-c58e7511ec39 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:57.163357 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496e2c6-72f0-4568-b2c1-c58e7511ec39 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:57.175106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3a86c-ca67-4c63-9ea5-d86b9bde7b0b None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:57.175646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3a86c-ca67-4c63-9ea5-d86b9bde7b0b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:57.175646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3a86c-ca67-4c63-9ea5-d86b9bde7b0b None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:57.177270 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 502/999] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:50:57.183444 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450f8f2-6196-4e65-b215-e9e588d00b1a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:57.183756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450f8f2-6196-4e65-b215-e9e588d00b1a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:57.183997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450f8f2-6196-4e65-b215-e9e588d00b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:57.185347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-a496e2c6-72f0-4568-b2c1-c58e7511ec39 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 16 14:50:57.185347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:57.185347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:57.185347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:57.185347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:57.185347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:57.185347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:57.185347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:57.185347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 16 14:50:57.185347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:57.185347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:57.185347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:57.185347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:57.185347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:57.185347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:57.185347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:57.185347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:57.186392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:57.186392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:57.186392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:57.186392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:57.186392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:57.186392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:57.186392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:57.186392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:57.186392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:57.186392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:57.186392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:57.186392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:57.186392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:57.186392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:57.186392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:57.186392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:57.187303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:57.187303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:57.187303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:57.187303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:57.187303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:57.187303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:57.187303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:57.187303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:57.187303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:57.187303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:57.187303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:57.187303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:57.187303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 16 14:50:57.187303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:57.188068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 498/1000] 10.222.0.27 () {56 vars in 1105 bytes} [Sat Jul 16 14:50:56 2022] GET /identity/v3/services/image => generated 86 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:57.195997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496e2c6-72f0-4568-b2c1-c58e7511ec39 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:57.209201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f450f8f2-6196-4e65-b215-e9e588d00b1a None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:57.212160 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f450f8f2-6196-4e65-b215-e9e588d00b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:57.295461 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 503/1001] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:50:57.319528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450f8f2-6196-4e65-b215-e9e588d00b1a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:57.333946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496e2c6-72f0-4568-b2c1-c58e7511ec39 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:57.356112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:57.384892 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20b883-e8a6-48f6-9f69-0c08a8f958a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:57.385879 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20b883-e8a6-48f6-9f69-0c08a8f958a0 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:57.386331 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20b883-e8a6-48f6-9f69-0c08a8f958a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:57.386609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20b883-e8a6-48f6-9f69-0c08a8f958a0 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:57.387344 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20b883-e8a6-48f6-9f69-0c08a8f958a0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:57.387652 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20b883-e8a6-48f6-9f69-0c08a8f958a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:57.389024 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20b883-e8a6-48f6-9f69-0c08a8f958a0 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:57.394639 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 499/1002] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:57.400979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20b883-e8a6-48f6-9f69-0c08a8f958a0 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:57.471487 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450f8f2-6196-4e65-b215-e9e588d00b1a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:57.475763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:57.524132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509a9fc-36b2-494c-ad33-d7e46fba9034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:57.525083 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509a9fc-36b2-494c-ad33-d7e46fba9034 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:57.525268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509a9fc-36b2-494c-ad33-d7e46fba9034 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:57.525489 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509a9fc-36b2-494c-ad33-d7e46fba9034 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:57.526142 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509a9fc-36b2-494c-ad33-d7e46fba9034 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:57.527555 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509a9fc-36b2-494c-ad33-d7e46fba9034 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:57.549340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-d509a9fc-36b2-494c-ad33-d7e46fba9034 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 14:50:57.549340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:57.549340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:57.549340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:57.549340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:57.549340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:57.549340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:57.549340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:57.549340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 14:50:57.549340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:57.549340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:57.549340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:57.549340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:57.549340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:57.549340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:57.549340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:57.549340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:57.550629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:57.550629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:57.550629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:57.550629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:57.550629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:57.550629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:57.550629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:57.550629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:57.550629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:57.550629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:57.550629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:57.550629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:57.550629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:57.550629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:57.550629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:57.550629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:57.552634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:57.552634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:57.552634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:57.552634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:57.552634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:57.552634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:57.552634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:57.552634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:57.552634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:57.552634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:57.552634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:57.552634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:57.552634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 16 14:50:57.552634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:50:57.552634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 504/1003] 10.222.0.27 () {56 vars in 1126 bytes} [Sat Jul 16 14:50:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:50:57.558303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509a9fc-36b2-494c-ad33-d7e46fba9034 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:57.561884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20b883-e8a6-48f6-9f69-0c08a8f958a0 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:57.612031 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b496e15-7edb-4cc1-92b3-df81f83a0997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:57.612783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b496e15-7edb-4cc1-92b3-df81f83a0997 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:57.613231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b496e15-7edb-4cc1-92b3-df81f83a0997 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:57.613516 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b496e15-7edb-4cc1-92b3-df81f83a0997 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:57.614540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b496e15-7edb-4cc1-92b3-df81f83a0997 None admin] RBAC: Adding query filter params (type=image) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:57.614852 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b496e15-7edb-4cc1-92b3-df81f83a0997 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:57.616496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b496e15-7edb-4cc1-92b3-df81f83a0997 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:57.626946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 500/1004] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:50:57 2022] GET /identity/v3/services?type=image => generated 122 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:57.745796 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509a9fc-36b2-494c-ad33-d7e46fba9034 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:57.820664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91dd0fe-c9c7-47c4-bfcd-08502e2d76bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:57.821743 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dd0fe-c9c7-47c4-bfcd-08502e2d76bb None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:57.822186 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dd0fe-c9c7-47c4-bfcd-08502e2d76bb None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:57.822526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91dd0fe-c9c7-47c4-bfcd-08502e2d76bb None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:57.823686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91dd0fe-c9c7-47c4-bfcd-08502e2d76bb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:57.824233 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91dd0fe-c9c7-47c4-bfcd-08502e2d76bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:57.826855 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91dd0fe-c9c7-47c4-bfcd-08502e2d76bb None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:57.837641 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 505/1005] 10.222.0.27 () {56 vars in 1122 bytes} [Sat Jul 16 14:50:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:57.846159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b496e15-7edb-4cc1-92b3-df81f83a0997 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:57.937185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:58.044012 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b496e15-7edb-4cc1-92b3-df81f83a0997 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:58.108153 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8777b2-9176-4a81-adb1-71bd882bfae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:58.109278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8777b2-9176-4a81-adb1-71bd882bfae2 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:58.109763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8777b2-9176-4a81-adb1-71bd882bfae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:58.110252 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8777b2-9176-4a81-adb1-71bd882bfae2 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:58.111549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8777b2-9176-4a81-adb1-71bd882bfae2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:58.112090 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8777b2-9176-4a81-adb1-71bd882bfae2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:58.114219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8777b2-9176-4a81-adb1-71bd882bfae2 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:58.125641 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 501/1006] 10.222.0.27 () {56 vars in 1122 bytes} [Sat Jul 16 14:50:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:58.134412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91dd0fe-c9c7-47c4-bfcd-08502e2d76bb None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:58.166350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ff909-6504-4f24-8523-186027d1f1b0 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:58.166835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ff909-6504-4f24-8523-186027d1f1b0 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:58.167185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ff909-6504-4f24-8523-186027d1f1b0 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:58.168794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 502/1007] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:50:58.175060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba58e3-d03f-4316-b9ff-b7ec29a970e0 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:58.175448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba58e3-d03f-4316-b9ff-b7ec29a970e0 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:58.175734 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba58e3-d03f-4316-b9ff-b7ec29a970e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:58.184598 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:58.203050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-1dba58e3-d03f-4316-b9ff-b7ec29a970e0 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:58.205134 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-1dba58e3-d03f-4316-b9ff-b7ec29a970e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:58.267943 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91dd0fe-c9c7-47c4-bfcd-08502e2d76bb None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:58.296262 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 503/1008] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:50:58.318230 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35178093-1c17-4e1f-bb9a-911c28d0576d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:58.319971 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35178093-1c17-4e1f-bb9a-911c28d0576d None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:58.320663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35178093-1c17-4e1f-bb9a-911c28d0576d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:58.321129 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35178093-1c17-4e1f-bb9a-911c28d0576d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:58.321367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba58e3-d03f-4316-b9ff-b7ec29a970e0 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:58.322571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35178093-1c17-4e1f-bb9a-911c28d0576d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:58.325056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35178093-1c17-4e1f-bb9a-911c28d0576d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:58.379717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:58.418710 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 506/1009] 10.222.0.27 () {60 vars in 1144 bytes} [Sat Jul 16 14:50:58 2022] POST /identity/v3/endpoints => generated 325 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:50:58.479806 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba58e3-d03f-4316-b9ff-b7ec29a970e0 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:58.551843 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad42da-3c47-43d0-a929-c7b883caaac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:58.552969 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad42da-3c47-43d0-a929-c7b883caaac6 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:58.553895 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad42da-3c47-43d0-a929-c7b883caaac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:58.554356 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad42da-3c47-43d0-a929-c7b883caaac6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:58.555483 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad42da-3c47-43d0-a929-c7b883caaac6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:58.557847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad42da-3c47-43d0-a929-c7b883caaac6 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:58.576619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-a1ad42da-3c47-43d0-a929-c7b883caaac6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 16 14:50:58.576619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:58.576619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:50:58.576619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:50:58.576619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:50:58.576619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:50:58.576619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:50:58.576619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:58.576619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 16 14:50:58.576619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:58.576619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:50:58.576619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:58.576619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:50:58.576619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:50:58.576619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:50:58.576619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:50:58.576619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:50:58.578311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:50:58.578311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:50:58.578311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:50:58.578311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:50:58.578311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:50:58.578311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:50:58.578311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:50:58.578311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:50:58.578311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:50:58.578311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:50:58.578311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:50:58.578311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:50:58.578311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:50:58.578311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:50:58.578311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:50:58.578311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:50:58.580112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:50:58.580112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:50:58.580112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:50:58.580112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:50:58.580112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:50:58.580112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:50:58.580112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:50:58.580112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:50:58.580112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:50:58.580112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:50:58.580112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:50:58.580112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:50:58.580112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 16 14:50:58.580112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:50:58.580112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 504/1010] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:50:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:50:58.588337 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35178093-1c17-4e1f-bb9a-911c28d0576d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:58.710370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:58.721391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35178093-1c17-4e1f-bb9a-911c28d0576d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:58.760951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aee3eb-2ac7-4695-8c30-639cf5189b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:58.761674 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aee3eb-2ac7-4695-8c30-639cf5189b3a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:58.761951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aee3eb-2ac7-4695-8c30-639cf5189b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:58.762225 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aee3eb-2ac7-4695-8c30-639cf5189b3a None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:58.763125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aee3eb-2ac7-4695-8c30-639cf5189b3a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:58.763395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aee3eb-2ac7-4695-8c30-639cf5189b3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:58.764720 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aee3eb-2ac7-4695-8c30-639cf5189b3a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:58.772606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 507/1011] 10.222.0.27 () {56 vars in 1112 bytes} [Sat Jul 16 14:50:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:50:58.778758 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad42da-3c47-43d0-a929-c7b883caaac6 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:58.959617 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:58.968032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad42da-3c47-43d0-a929-c7b883caaac6 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:59.028285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37d968-a454-4eed-b9af-f18437a301ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:59.029409 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37d968-a454-4eed-b9af-f18437a301ce None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:59.029892 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37d968-a454-4eed-b9af-f18437a301ce None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:59.030373 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37d968-a454-4eed-b9af-f18437a301ce None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:59.031730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37d968-a454-4eed-b9af-f18437a301ce None admin] RBAC: Adding query filter params (type=compute) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:50:59.032277 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37d968-a454-4eed-b9af-f18437a301ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:59.034791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37d968-a454-4eed-b9af-f18437a301ce None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:59.043965 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 505/1012] 10.222.0.27 () {56 vars in 1112 bytes} [Sat Jul 16 14:50:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:50:59.052807 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aee3eb-2ac7-4695-8c30-639cf5189b3a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:50:59.231317 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aee3eb-2ac7-4695-8c30-639cf5189b3a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:50:59.291354 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae0e7f-5d6c-45ea-a544-4fadd5593dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:50:59.292473 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae0e7f-5d6c-45ea-a544-4fadd5593dae None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:59.292954 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae0e7f-5d6c-45ea-a544-4fadd5593dae None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:59.293431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae0e7f-5d6c-45ea-a544-4fadd5593dae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:59.294837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae0e7f-5d6c-45ea-a544-4fadd5593dae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:50:59.297431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae0e7f-5d6c-45ea-a544-4fadd5593dae None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:50:59.423440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 508/1013] 10.222.0.27 () {60 vars in 1144 bytes} [Sat Jul 16 14:50:59 2022] POST /identity/v3/endpoints => generated 334 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:50:59.752562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3720cf9-721b-4877-b022-1603aa10f633 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:59.756681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3720cf9-721b-4877-b022-1603aa10f633 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:59.756681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3720cf9-721b-4877-b022-1603aa10f633 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:59.756681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 506/1014] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:50:59.760784 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ea95a-8c00-4bb2-b44a-57968c646f14 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:50:59.761102 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ea95a-8c00-4bb2-b44a-57968c646f14 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:50:59.761380 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ea95a-8c00-4bb2-b44a-57968c646f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:50:59.783234 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5c6ea95a-8c00-4bb2-b44a-57968c646f14 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:50:59.784699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5c6ea95a-8c00-4bb2-b44a-57968c646f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:50:59.796394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:50:59.886030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 509/1015] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:50:59 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:50:59.909107 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3720cf9-721b-4877-b022-1603aa10f633 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:00.032312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3720cf9-721b-4877-b022-1603aa10f633 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:00.072764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19603c69-9068-4d61-81dc-35bd275b4376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:00.073797 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19603c69-9068-4d61-81dc-35bd275b4376 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:00.074278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19603c69-9068-4d61-81dc-35bd275b4376 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:00.074750 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19603c69-9068-4d61-81dc-35bd275b4376 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:00.075710 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19603c69-9068-4d61-81dc-35bd275b4376 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:00.077318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19603c69-9068-4d61-81dc-35bd275b4376 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:00.154916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 507/1016] 10.222.0.27 () {60 vars in 1141 bytes} [Sat Jul 16 14:50:59 2022] POST /identity/v3/services => generated 242 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:51:00.601744 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa29abd-ac28-4fca-97ce-fec99b65ff69 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:00.602300 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa29abd-ac28-4fca-97ce-fec99b65ff69 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:00.602809 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa29abd-ac28-4fca-97ce-fec99b65ff69 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:00.604946 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 510/1017] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:00.610917 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad0c9d-77f9-4e31-adee-b9e17739e2d2 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:00.611217 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad0c9d-77f9-4e31-adee-b9e17739e2d2 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:00.611481 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad0c9d-77f9-4e31-adee-b9e17739e2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:00.632588 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-c0ad0c9d-77f9-4e31-adee-b9e17739e2d2 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:00.634324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c0ad0c9d-77f9-4e31-adee-b9e17739e2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:00.663329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:00.740294 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 508/1018] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:51:00.765039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa29abd-ac28-4fca-97ce-fec99b65ff69 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:00.950449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa29abd-ac28-4fca-97ce-fec99b65ff69 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:01.014862 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f318e5-f620-43e1-8241-584cede9b90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:01.016265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f318e5-f620-43e1-8241-584cede9b90a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:01.016736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f318e5-f620-43e1-8241-584cede9b90a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:01.017148 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f318e5-f620-43e1-8241-584cede9b90a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:01.019243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f318e5-f620-43e1-8241-584cede9b90a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:01.022494 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f318e5-f620-43e1-8241-584cede9b90a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:01.030820 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 511/1019] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:51:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:51:01.033235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad0c9d-77f9-4e31-adee-b9e17739e2d2 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:01.173405 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad0c9d-77f9-4e31-adee-b9e17739e2d2 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:01.223384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac43828-7934-46eb-9a28-411a5398b92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:01.224257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac43828-7934-46eb-9a28-411a5398b92d None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:01.224545 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac43828-7934-46eb-9a28-411a5398b92d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:01.224813 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac43828-7934-46eb-9a28-411a5398b92d None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:01.226450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac43828-7934-46eb-9a28-411a5398b92d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:01.228038 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac43828-7934-46eb-9a28-411a5398b92d None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:01.330969 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:01.347323 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 509/1020] 10.222.0.27 () {60 vars in 1141 bytes} [Sat Jul 16 14:51:01 2022] POST /identity/v3/projects => generated 311 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:51:01.641972 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70831789-78b6-4da4-808b-df1aaf8450fa None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:01.642789 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70831789-78b6-4da4-808b-df1aaf8450fa None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:01.642789 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70831789-78b6-4da4-808b-df1aaf8450fa None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:01.644079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 512/1021] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:01.649945 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f980a07-2fc8-4781-89f5-bd171b900b5c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:01.650233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f980a07-2fc8-4781-89f5-bd171b900b5c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:01.650515 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f980a07-2fc8-4781-89f5-bd171b900b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:01.668794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-7f980a07-2fc8-4781-89f5-bd171b900b5c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:01.670001 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7f980a07-2fc8-4781-89f5-bd171b900b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:01.744308 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 510/1022] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:51:01.767200 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70831789-78b6-4da4-808b-df1aaf8450fa None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:01.778405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:01.932106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70831789-78b6-4da4-808b-df1aaf8450fa None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:01.974823 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3f154-2c04-482a-bc6b-e21933515abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:01.975742 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3f154-2c04-482a-bc6b-e21933515abd None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:01.976025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3f154-2c04-482a-bc6b-e21933515abd None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:01.976300 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3f154-2c04-482a-bc6b-e21933515abd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:01.988952 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c3f154-2c04-482a-bc6b-e21933515abd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:51:01.997414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-68c3f154-2c04-482a-bc6b-e21933515abd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 14:51:01.997414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:01.997414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:01.997414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:01.997414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:01.997414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:01.997414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:01.997414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:01.997414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:01.997414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:01.997414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:01.997414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:01.997414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:51:01.997414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:51:01.997414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:51:01.997414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:51:01.998713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:51:01.998713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:51:01.998713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:51:01.998713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:51:01.998713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:01.998713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:01.998713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:01.998713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:01.998713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:01.998713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:01.998713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:01.998713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:01.998713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:01.998713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:01.998713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:01.998713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:02.004435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:02.004435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:02.004435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:02.004435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:02.004435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:51:02.004435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:51:02.004435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:51:02.004435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:51:02.004435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:51:02.004435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:51:02.004435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 14:51:02.004435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:51:02.004435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 513/1023] 10.222.0.27 () {56 vars in 1120 bytes} [Sat Jul 16 14:51:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:51:02.008123 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f980a07-2fc8-4781-89f5-bd171b900b5c None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:02.122089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f980a07-2fc8-4781-89f5-bd171b900b5c None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:02.162255 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:02.171194 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d1a65-d7b5-4dd9-b8be-c4bfbf641ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:02.172087 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d1a65-d7b5-4dd9-b8be-c4bfbf641ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:02.177076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d1a65-d7b5-4dd9-b8be-c4bfbf641ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:02.177076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d1a65-d7b5-4dd9-b8be-c4bfbf641ba0 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:02.177076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d1a65-d7b5-4dd9-b8be-c4bfbf641ba0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:02.177076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d1a65-d7b5-4dd9-b8be-c4bfbf641ba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:02.177076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d1a65-d7b5-4dd9-b8be-c4bfbf641ba0 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:02.187792 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 511/1024] 10.222.0.27 () {56 vars in 1115 bytes} [Sat Jul 16 14:51:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:02.195268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3f154-2c04-482a-bc6b-e21933515abd None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:02.310641 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:02.374021 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3f154-2c04-482a-bc6b-e21933515abd None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:02.433697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5483de-a503-403c-b3be-363a4b57c598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:02.434996 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5483de-a503-403c-b3be-363a4b57c598 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:02.435505 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5483de-a503-403c-b3be-363a4b57c598 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:02.436005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5483de-a503-403c-b3be-363a4b57c598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:02.455315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5483de-a503-403c-b3be-363a4b57c598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:02.455574 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c023ee-d673-466b-89b7-684e1b7b5151 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:02.455574 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c023ee-d673-466b-89b7-684e1b7b5151 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:02.455900 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c023ee-d673-466b-89b7-684e1b7b5151 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:02.457393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 512/1025] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:51:02.457503 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5483de-a503-403c-b3be-363a4b57c598 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:02.464069 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695201c-b7ed-478b-8bf2-e64a5e35d772 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:02.464321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695201c-b7ed-478b-8bf2-e64a5e35d772 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:02.464597 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695201c-b7ed-478b-8bf2-e64a5e35d772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:02.482013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-6b5483de-a503-403c-b3be-363a4b57c598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:51:02.482013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:02.482013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:02.482013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:02.482013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:02.482013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:02.482013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:02.482013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:02.482013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:02.482013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:02.482013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:02.482013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:02.482013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:51:02.482013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:51:02.482013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:51:02.484169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:51:02.484169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:02.484169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:02.484169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:02.484169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:02.484169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:02.484169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:02.484169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:02.484169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:02.484169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:02.484169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:02.484169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:02.484169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:02.484169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:02.484169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:02.484169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:02.484169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:02.485929 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:51:02.485929 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:51:02.485929 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:51:02.485929 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:51:02.485929 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 514/1026] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:51:02.488086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-e695201c-b7ed-478b-8bf2-e64a5e35d772 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:02.489569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e695201c-b7ed-478b-8bf2-e64a5e35d772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:02.492885 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5483de-a503-403c-b3be-363a4b57c598 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:02.577908 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 513/1027] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:51:02.600662 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695201c-b7ed-478b-8bf2-e64a5e35d772 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:02.645632 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5483de-a503-403c-b3be-363a4b57c598 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:02.669581 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:02.697960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5d08c-db97-44e3-8559-b1b207bf4213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:02.699027 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5d08c-db97-44e3-8559-b1b207bf4213 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:02.699488 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5d08c-db97-44e3-8559-b1b207bf4213 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:02.699851 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5d08c-db97-44e3-8559-b1b207bf4213 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:02.700952 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5d08c-db97-44e3-8559-b1b207bf4213 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:02.701456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5d08c-db97-44e3-8559-b1b207bf4213 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:02.703762 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5d08c-db97-44e3-8559-b1b207bf4213 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:02.726946 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 515/1028] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:51:02.731190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5d08c-db97-44e3-8559-b1b207bf4213 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:02.758917 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695201c-b7ed-478b-8bf2-e64a5e35d772 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:02.789434 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:02.806426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a5914-901a-4ace-a3b0-cfc178c9350f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:02.807587 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a5914-901a-4ace-a3b0-cfc178c9350f None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:02.807938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a5914-901a-4ace-a3b0-cfc178c9350f None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:02.808273 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a5914-901a-4ace-a3b0-cfc178c9350f None admin] PATH_INFO: `/v3/services/image` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:02.809250 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a5914-901a-4ace-a3b0-cfc178c9350f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:02.810811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a5914-901a-4ace-a3b0-cfc178c9350f None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:02.827073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-db4a5914-901a-4ace-a3b0-cfc178c9350f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 16 14:51:02.827073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:02.827073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:51:02.827073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:51:02.827073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:51:02.827073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:51:02.827073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:51:02.827073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:51:02.827073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 16 14:51:02.827073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:51:02.827073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:51:02.827073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:51:02.827073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:02.827073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:02.827073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:02.827073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:02.827073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:02.828428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:02.828428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:02.828428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:02.828428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:02.828428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:02.828428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:02.828428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:51:02.828428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:51:02.828428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:51:02.828428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:51:02.828428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:02.828428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:02.828428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:02.828428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:02.828428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:02.828428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:02.829423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:02.829423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:02.829423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:02.829423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:02.829423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:02.829423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:02.829423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:02.829423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:02.829423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:02.829423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:02.829423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:51:02.829423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:51:02.829423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 16 14:51:02.829423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:51:02.835177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 514/1029] 10.222.0.27 () {56 vars in 1105 bytes} [Sat Jul 16 14:51:02 2022] GET /identity/v3/services/image => generated 86 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:51:02.838298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a5914-901a-4ace-a3b0-cfc178c9350f None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:02.868740 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5d08c-db97-44e3-8559-b1b207bf4213 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:02.915034 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f4589-2423-4c2b-bc8e-4bec6ab7f00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:02.915840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f4589-2423-4c2b-bc8e-4bec6ab7f00f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:02.916118 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f4589-2423-4c2b-bc8e-4bec6ab7f00f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:02.916386 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f4589-2423-4c2b-bc8e-4bec6ab7f00f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:02.925328 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f4589-2423-4c2b-bc8e-4bec6ab7f00f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:02.926206 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f4589-2423-4c2b-bc8e-4bec6ab7f00f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:02.947920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-7b5f4589-2423-4c2b-bc8e-4bec6ab7f00f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 16 14:51:02.947920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:02.947920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:02.947920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:02.947920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:02.947920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:02.947920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:02.947920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:02.947920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:02.947920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:02.947920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:02.947920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:02.947920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:51:02.947920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:51:02.947920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:51:02.947920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:51:02.949704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:02.949704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:02.949704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:51:02.949704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:51:02.949704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:51:02.949704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:51:02.949704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:02.949704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:02.949704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:02.949704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:02.949704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:02.949704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:02.949704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:02.949704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:02.949704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:02.949704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:02.949704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:02.949704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:02.950822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:02.950822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:02.950822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:51:02.950822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:51:02.950822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:51:02.950822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:51:02.950822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:51:02.950822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:51:02.950822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 16 14:51:02.950822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:51:02.950822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 516/1030] 10.222.0.27 () {56 vars in 1093 bytes} [Sat Jul 16 14:51:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:51:02.958607 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f4589-2423-4c2b-bc8e-4bec6ab7f00f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:02.995851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a5914-901a-4ace-a3b0-cfc178c9350f None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:03.021475 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:03.036736 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f4c18-6d75-4b0e-8b91-5a92b56aea60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:03.037514 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f4c18-6d75-4b0e-8b91-5a92b56aea60 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:03.037865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f4c18-6d75-4b0e-8b91-5a92b56aea60 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:03.038323 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f4c18-6d75-4b0e-8b91-5a92b56aea60 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:03.039293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f4c18-6d75-4b0e-8b91-5a92b56aea60 None admin] RBAC: Adding query filter params (name=image) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:03.039601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f4c18-6d75-4b0e-8b91-5a92b56aea60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:03.040985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f4c18-6d75-4b0e-8b91-5a92b56aea60 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:03.046850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 515/1031] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:51:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:51:03.052432 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f4c18-6d75-4b0e-8b91-5a92b56aea60 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:03.100518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f4589-2423-4c2b-bc8e-4bec6ab7f00f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:03.124107 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:03.151570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88111155-3da1-4659-9135-ad0b9a72fab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:03.152342 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88111155-3da1-4659-9135-ad0b9a72fab4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:03.152625 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88111155-3da1-4659-9135-ad0b9a72fab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:03.152894 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88111155-3da1-4659-9135-ad0b9a72fab4 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:03.153751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88111155-3da1-4659-9135-ad0b9a72fab4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:03.154054 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88111155-3da1-4659-9135-ad0b9a72fab4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:03.155551 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88111155-3da1-4659-9135-ad0b9a72fab4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:03.177374 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 517/1032] 10.222.0.27 () {56 vars in 1133 bytes} [Sat Jul 16 14:51:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:03.184923 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88111155-3da1-4659-9135-ad0b9a72fab4 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:03.197226 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f4c18-6d75-4b0e-8b91-5a92b56aea60 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:03.242123 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de688a-7239-433d-a7ae-d0f642b838c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:03.243101 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de688a-7239-433d-a7ae-d0f642b838c6 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:03.243391 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de688a-7239-433d-a7ae-d0f642b838c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:03.243675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de688a-7239-433d-a7ae-d0f642b838c6 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:03.244532 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de688a-7239-433d-a7ae-d0f642b838c6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:03.244843 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de688a-7239-433d-a7ae-d0f642b838c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:03.246220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de688a-7239-433d-a7ae-d0f642b838c6 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:03.262437 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 516/1033] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:51:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:51:03.264530 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de688a-7239-433d-a7ae-d0f642b838c6 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:03.356248 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:03.363150 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88111155-3da1-4659-9135-ad0b9a72fab4 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:03.406899 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179df0ce-97ed-4dcd-ab3a-474b744fa5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:03.408008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179df0ce-97ed-4dcd-ab3a-474b744fa5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:03.408507 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179df0ce-97ed-4dcd-ab3a-474b744fa5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:03.409004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179df0ce-97ed-4dcd-ab3a-474b744fa5e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:03.410289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de688a-7239-433d-a7ae-d0f642b838c6 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:03.436863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:03.468741 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbedfb-8048-4365-8e21-81398d973cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:03.469404 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbedfb-8048-4365-8e21-81398d973cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:03.469686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbedfb-8048-4365-8e21-81398d973cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:03.469963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbedfb-8048-4365-8e21-81398d973cb5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:03.470767 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbedfb-8048-4365-8e21-81398d973cb5 None admin] RBAC: Adding query filter params (interface=public, service_id=70075c2d66b740c896e78433bf9d7d99, region_id=RegionOne) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:03.471105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbedfb-8048-4365-8e21-81398d973cb5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:03.472484 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbedfb-8048-4365-8e21-81398d973cb5 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:03.484469 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 517/1034] 10.222.0.27 () {56 vars in 1251 bytes} [Sat Jul 16 14:51:03 2022] GET /identity/v3/endpoints?service_id=70075c2d66b740c896e78433bf9d7d99&interface=public®ion_id=RegionOne => generated 194 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:51:03.485823 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbedfb-8048-4365-8e21-81398d973cb5 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:03.525459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179df0ce-97ed-4dcd-ab3a-474b744fa5e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:03.526866 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179df0ce-97ed-4dcd-ab3a-474b744fa5e4 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:03.634555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbedfb-8048-4365-8e21-81398d973cb5 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:03.641119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:03.648822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 518/1035] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:51:03 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 16 14:51:03.650031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5dbc9d-423e-4492-a6c2-81d5a5655f25 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:03.650327 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5dbc9d-423e-4492-a6c2-81d5a5655f25 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:03.650604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5dbc9d-423e-4492-a6c2-81d5a5655f25 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:03.651926 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 519/1036] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:03.654744 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5dbc9d-423e-4492-a6c2-81d5a5655f25 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:03.679883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba851b62-ded2-4199-ba32-59e8bc5ceee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:03.680560 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba851b62-ded2-4199-ba32-59e8bc5ceee2 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:03.680838 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba851b62-ded2-4199-ba32-59e8bc5ceee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:03.681109 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba851b62-ded2-4199-ba32-59e8bc5ceee2 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:03.681818 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba851b62-ded2-4199-ba32-59e8bc5ceee2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:03.683236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba851b62-ded2-4199-ba32-59e8bc5ceee2 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:03.690166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 518/1037] 10.222.0.27 () {56 vars in 1087 bytes} [Sat Jul 16 14:51:03 2022] GET /identity/v3/services => generated 1957 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 14:51:03.691480 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab11926-df0d-42cf-8b91-003ce7d35fef None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:03.691764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab11926-df0d-42cf-8b91-003ce7d35fef None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:03.692038 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab11926-df0d-42cf-8b91-003ce7d35fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:03.711015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-aab11926-df0d-42cf-8b91-003ce7d35fef None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:03.712741 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-aab11926-df0d-42cf-8b91-003ce7d35fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:03.763810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:03.818242 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 519/1038] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:51:03.823879 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5dbc9d-423e-4492-a6c2-81d5a5655f25 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:03.842949 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab11926-df0d-42cf-8b91-003ce7d35fef None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:03.888146 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c3086-9d0e-435a-8972-f8083125c903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:03.888948 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c3086-9d0e-435a-8972-f8083125c903 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:03.889221 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c3086-9d0e-435a-8972-f8083125c903 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:03.889491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c3086-9d0e-435a-8972-f8083125c903 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:03.909285 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c3086-9d0e-435a-8972-f8083125c903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:03.912088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c3086-9d0e-435a-8972-f8083125c903 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:03.937697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-486c3086-9d0e-435a-8972-f8083125c903 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:51:03.937697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:03.937697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:03.937697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:03.937697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:03.937697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:03.937697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:03.937697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:03.937697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:03.937697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:03.937697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:03.937697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:03.937697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:51:03.937697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:51:03.937697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:51:03.938669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:51:03.938669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:03.938669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:03.938669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:03.938669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:03.938669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:03.938669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:03.938669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:03.938669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:03.938669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:03.938669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:03.938669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:03.938669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:03.938669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:03.938669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:03.938669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:03.938669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:03.939613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:51:03.939613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:51:03.939613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:51:03.939613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:51:03.939613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 520/1039] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:51:03.946947 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c3086-9d0e-435a-8972-f8083125c903 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:03.981861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab11926-df0d-42cf-8b91-003ce7d35fef None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:04.013286 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:04.023332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a1d4e-bce0-45ea-b0da-c7331e3b9e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:04.024189 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a1d4e-bce0-45ea-b0da-c7331e3b9e02 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:04.024671 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a1d4e-bce0-45ea-b0da-c7331e3b9e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:04.024952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a1d4e-bce0-45ea-b0da-c7331e3b9e02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:04.026278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a1d4e-bce0-45ea-b0da-c7331e3b9e02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:04.027778 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a1d4e-bce0-45ea-b0da-c7331e3b9e02 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:04.029632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 520/1040] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:51:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:51:04.035529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a1d4e-bce0-45ea-b0da-c7331e3b9e02 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:04.084049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c3086-9d0e-435a-8972-f8083125c903 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:04.113107 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:04.144209 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc7b73-8155-4c63-8834-8151d802647f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:04.144948 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc7b73-8155-4c63-8834-8151d802647f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:04.145234 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc7b73-8155-4c63-8834-8151d802647f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:04.145735 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc7b73-8155-4c63-8834-8151d802647f None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:04.146473 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc7b73-8155-4c63-8834-8151d802647f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:04.147470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc7b73-8155-4c63-8834-8151d802647f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:04.148914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc7b73-8155-4c63-8834-8151d802647f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:04.165059 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 521/1041] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:51:04.170618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc7b73-8155-4c63-8834-8151d802647f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:04.197232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a1d4e-bce0-45ea-b0da-c7331e3b9e02 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:04.260201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321828f-761b-4639-801f-c898b2152cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:04.261314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321828f-761b-4639-801f-c898b2152cd6 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:04.261770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321828f-761b-4639-801f-c898b2152cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:04.262227 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321828f-761b-4639-801f-c898b2152cd6 None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:04.263549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a321828f-761b-4639-801f-c898b2152cd6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:04.265943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a321828f-761b-4639-801f-c898b2152cd6 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:04.314332 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc7b73-8155-4c63-8834-8151d802647f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:04.325790 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:04.358610 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91c804-1c29-4fca-b92a-f28a692dca2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:04.359469 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91c804-1c29-4fca-b92a-f28a692dca2c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:04.359767 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91c804-1c29-4fca-b92a-f28a692dca2c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:04.360033 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91c804-1c29-4fca-b92a-f28a692dca2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:04.368663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a91c804-1c29-4fca-b92a-f28a692dca2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:04.370169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a91c804-1c29-4fca-b92a-f28a692dca2c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:04.386557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-5a91c804-1c29-4fca-b92a-f28a692dca2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:51:04.386557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:04.386557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:04.386557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:04.386557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:04.386557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:04.386557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:04.386557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:04.386557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:04.386557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:04.386557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:04.386557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:04.386557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:51:04.386557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:51:04.386557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:51:04.387810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:51:04.387810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:04.387810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:04.387810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:04.387810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:04.387810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:04.387810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:04.387810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:04.387810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:04.387810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:04.387810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:04.387810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:04.387810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:04.387810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:04.387810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:04.387810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:04.387810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:04.388981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:51:04.388981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:51:04.388981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:51:04.388981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:51:04.388981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:51:04.388981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:51:04.388981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:51:04.388981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:51:04.388981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 522/1042] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:51:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:51:04.394848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 521/1043] 10.222.0.27 () {60 vars in 1132 bytes} [Sat Jul 16 14:51:04 2022] POST /identity/v3/users => generated 285 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 14:51:04.397493 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91c804-1c29-4fca-b92a-f28a692dca2c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:04.576487 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91c804-1c29-4fca-b92a-f28a692dca2c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:04.629391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac9e70-d6b4-431a-89e5-8614b2f607c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:04.630344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac9e70-d6b4-431a-89e5-8614b2f607c7 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:04.630814 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac9e70-d6b4-431a-89e5-8614b2f607c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:04.631457 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac9e70-d6b4-431a-89e5-8614b2f607c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:04.632815 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac9e70-d6b4-431a-89e5-8614b2f607c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:04.633307 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac9e70-d6b4-431a-89e5-8614b2f607c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:04.635793 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac9e70-d6b4-431a-89e5-8614b2f607c7 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:04.655190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 523/1044] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:51:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:04.661378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321828f-761b-4639-801f-c898b2152cd6 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:04.783883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:04.791389 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321828f-761b-4639-801f-c898b2152cd6 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:04.830248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6b1ab-b043-4f43-a7f6-5658ffa4ae83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:04.830936 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6b1ab-b043-4f43-a7f6-5658ffa4ae83 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:04.831223 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6b1ab-b043-4f43-a7f6-5658ffa4ae83 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:04.831530 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6b1ab-b043-4f43-a7f6-5658ffa4ae83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:04.832382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6b1ab-b043-4f43-a7f6-5658ffa4ae83 None admin] RBAC: Adding query filter params (role.id=e3c9141bd01741b59c8701275be9154c, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=3f2e241494244edf8d283e8f44192c46) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:04.832691 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6b1ab-b043-4f43-a7f6-5658ffa4ae83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:04.834687 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6b1ab-b043-4f43-a7f6-5658ffa4ae83 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:04.841997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 522/1045] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:51:04 2022] GET /identity/v3/role_assignments?user.id=3f2e241494244edf8d283e8f44192c46&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=e3c9141bd01741b59c8701275be9154c => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:05.817745 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da6fc8-8c21-4914-9ac7-418dc6358a00 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:05.818318 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da6fc8-8c21-4914-9ac7-418dc6358a00 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:05.818318 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da6fc8-8c21-4914-9ac7-418dc6358a00 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:05.820256 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 524/1046] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:05.826189 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f3447-f430-447c-a1f8-6a624cc04850 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:05.826676 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f3447-f430-447c-a1f8-6a624cc04850 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:05.826865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f3447-f430-447c-a1f8-6a624cc04850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:05.844977 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-bf2f3447-f430-447c-a1f8-6a624cc04850 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:05.846246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-bf2f3447-f430-447c-a1f8-6a624cc04850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:05.950759 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 523/1047] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:51:05.992319 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da6fc8-8c21-4914-9ac7-418dc6358a00 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:06.037182 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:06.125931 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da6fc8-8c21-4914-9ac7-418dc6358a00 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:06.164468 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d70ae3-53e9-4287-8762-790029086f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:06.165262 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d70ae3-53e9-4287-8762-790029086f44 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:06.165542 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d70ae3-53e9-4287-8762-790029086f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:06.165816 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d70ae3-53e9-4287-8762-790029086f44 None admin] PATH_INFO: `/v3/services/image` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:06.166471 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d70ae3-53e9-4287-8762-790029086f44 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:06.167877 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d70ae3-53e9-4287-8762-790029086f44 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:06.182163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-36d70ae3-53e9-4287-8762-790029086f44 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 16 14:51:06.182163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:06.182163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:51:06.182163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:51:06.182163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:51:06.182163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:51:06.182163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:51:06.182163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:51:06.182163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 16 14:51:06.182163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:51:06.182163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:51:06.182163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:51:06.182163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:06.182163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:06.182163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:06.182163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:06.182163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:06.183603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:06.183603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:06.183603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:06.183603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:06.183603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:06.183603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:06.183603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:51:06.183603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:51:06.183603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:51:06.183603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:51:06.183603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:06.183603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:06.183603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:06.183603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:06.183603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:06.183603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:06.184950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:06.184950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:06.184950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:06.184950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:06.184950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:06.184950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:06.184950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:06.184950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:06.184950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:06.184950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:06.184950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:51:06.184950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:51:06.184950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 16 14:51:06.184950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:51:06.184950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 525/1048] 10.222.0.27 () {56 vars in 1105 bytes} [Sat Jul 16 14:51:05 2022] GET /identity/v3/services/image => generated 86 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:51:06.194993 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f3447-f430-447c-a1f8-6a624cc04850 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:06.294714 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:06.312852 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f3447-f430-447c-a1f8-6a624cc04850 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:06.359517 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392e379-deaf-44de-8f74-b417375b161d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:06.360218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392e379-deaf-44de-8f74-b417375b161d None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:06.360504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392e379-deaf-44de-8f74-b417375b161d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:06.360781 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392e379-deaf-44de-8f74-b417375b161d None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:06.361563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392e379-deaf-44de-8f74-b417375b161d None admin] RBAC: Adding query filter params (name=image) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:06.361870 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392e379-deaf-44de-8f74-b417375b161d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:06.363276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392e379-deaf-44de-8f74-b417375b161d None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:06.374915 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 524/1049] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:51:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:51:06.377601 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d70ae3-53e9-4287-8762-790029086f44 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:06.524573 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:06.540006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d70ae3-53e9-4287-8762-790029086f44 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:06.599493 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fc754-5a22-4f66-b394-6f20bf66ca8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:06.601103 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fc754-5a22-4f66-b394-6f20bf66ca8c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:06.601734 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fc754-5a22-4f66-b394-6f20bf66ca8c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:06.602238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fc754-5a22-4f66-b394-6f20bf66ca8c None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:06.603800 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463fc754-5a22-4f66-b394-6f20bf66ca8c None admin] RBAC: Adding query filter params (type=image) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:06.604461 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463fc754-5a22-4f66-b394-6f20bf66ca8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:06.607238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463fc754-5a22-4f66-b394-6f20bf66ca8c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:06.623235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 526/1050] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:51:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:06.625499 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392e379-deaf-44de-8f74-b417375b161d None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:06.774409 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392e379-deaf-44de-8f74-b417375b161d None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:06.816389 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9c8a8-ee65-4646-af3d-9b42e513e958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:06.817215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9c8a8-ee65-4646-af3d-9b42e513e958 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:06.817601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9c8a8-ee65-4646-af3d-9b42e513e958 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:06.817998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9c8a8-ee65-4646-af3d-9b42e513e958 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:06.819484 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9c8a8-ee65-4646-af3d-9b42e513e958 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:06.821039 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9c8a8-ee65-4646-af3d-9b42e513e958 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:06.912080 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 525/1051] 10.222.0.27 () {60 vars in 1144 bytes} [Sat Jul 16 14:51:06 2022] POST /identity/v3/endpoints => generated 327 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:51:07.073770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce746d-c599-49bc-a0d0-43e2dd095bd5 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:07.074569 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce746d-c599-49bc-a0d0-43e2dd095bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:07.074569 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce746d-c599-49bc-a0d0-43e2dd095bd5 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:07.076028 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 527/1052] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:07.081529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2767268-e4e6-4243-834d-5280f23c60dc None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:07.081791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2767268-e4e6-4243-834d-5280f23c60dc None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:07.082081 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2767268-e4e6-4243-834d-5280f23c60dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:07.101690 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-a2767268-e4e6-4243-834d-5280f23c60dc None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:07.102921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a2767268-e4e6-4243-834d-5280f23c60dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:07.106973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:07.194175 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 526/1053] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:51:07.217297 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce746d-c599-49bc-a0d0-43e2dd095bd5 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:07.249346 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a75acc-667d-4a09-9a3c-bcaefce021c1 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:07.249639 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a75acc-667d-4a09-9a3c-bcaefce021c1 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:07.249938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a75acc-667d-4a09-9a3c-bcaefce021c1 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:07.251385 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 527/1054] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:51:07.257658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87eb40-76e9-43b2-a760-b8eb965f95bf None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:07.257945 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87eb40-76e9-43b2-a760-b8eb965f95bf None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:07.258233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87eb40-76e9-43b2-a760-b8eb965f95bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:07.277910 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-2b87eb40-76e9-43b2-a760-b8eb965f95bf None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:07.279467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-2b87eb40-76e9-43b2-a760-b8eb965f95bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:07.338982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce746d-c599-49bc-a0d0-43e2dd095bd5 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:07.380015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45feb40b-28eb-49fc-b3f0-4d04e4e5588c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:07.380831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45feb40b-28eb-49fc-b3f0-4d04e4e5588c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:07.381767 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45feb40b-28eb-49fc-b3f0-4d04e4e5588c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:07.382045 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45feb40b-28eb-49fc-b3f0-4d04e4e5588c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:07.393884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 528/1055] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:51:07.399236 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45feb40b-28eb-49fc-b3f0-4d04e4e5588c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:51:07.403099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-45feb40b-28eb-49fc-b3f0-4d04e4e5588c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:51:07.403099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:07.403099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:07.403099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:07.403099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:07.403099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:07.403099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:07.403099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:07.403099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:07.403099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:07.403099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:07.403099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:07.403099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:51:07.403099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:51:07.403099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:51:07.403099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:51:07.404870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:51:07.404870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:51:07.404870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:51:07.404870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:51:07.404870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:07.404870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:07.404870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:07.404870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:07.404870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:07.404870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:07.404870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:07.404870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:07.404870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:07.404870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:07.404870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:07.404870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:07.406501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:07.406501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:07.406501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:07.406501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:07.406501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:51:07.406501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:51:07.406501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:51:07.406501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:51:07.406501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:51:07.406501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:51:07.406501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:51:07.406501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:51:07.406501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 528/1056] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:51:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:51:07.411598 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87eb40-76e9-43b2-a760-b8eb965f95bf None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:07.420981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45feb40b-28eb-49fc-b3f0-4d04e4e5588c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:07.447430 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:07.542414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:07.558573 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45feb40b-28eb-49fc-b3f0-4d04e4e5588c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:07.588365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87eb40-76e9-43b2-a760-b8eb965f95bf None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:07.619180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe71c7-5725-4df1-8447-8b07c60ce472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:07.620481 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe71c7-5725-4df1-8447-8b07c60ce472 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:07.620932 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe71c7-5725-4df1-8447-8b07c60ce472 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:07.621433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe71c7-5725-4df1-8447-8b07c60ce472 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:07.639731 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe71c7-5725-4df1-8447-8b07c60ce472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:51:07.647753 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dc25e-e0f0-48a5-ae02-ec6c06cbfe7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:07.649097 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dc25e-e0f0-48a5-ae02-ec6c06cbfe7a None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:07.649786 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dc25e-e0f0-48a5-ae02-ec6c06cbfe7a None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:07.650417 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dc25e-e0f0-48a5-ae02-ec6c06cbfe7a None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:07.651914 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9dc25e-e0f0-48a5-ae02-ec6c06cbfe7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:07.652646 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9dc25e-e0f0-48a5-ae02-ec6c06cbfe7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:07.655188 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9dc25e-e0f0-48a5-ae02-ec6c06cbfe7a None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:07.655527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-f7fe71c7-5725-4df1-8447-8b07c60ce472 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 14:51:07.655527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:07.655527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:07.655527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:07.655527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:07.655527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:07.655527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:07.655527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:07.655527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:07.655527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:07.655527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:07.655527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:07.655527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:51:07.655527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:51:07.655527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:51:07.655527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:51:07.657491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:51:07.657491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:51:07.657491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:51:07.657491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:51:07.657491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:07.657491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:07.657491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:07.657491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:07.657491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:07.657491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:07.657491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:07.657491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:07.657491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:07.657491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:07.657491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:07.657491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:07.658687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:07.658687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:07.658687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:07.658687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:07.658687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:51:07.658687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:51:07.658687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:51:07.658687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:51:07.658687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:51:07.658687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:51:07.658687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 14:51:07.658687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:51:07.658687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 529/1057] 10.222.0.27 () {56 vars in 1120 bytes} [Sat Jul 16 14:51:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:51:07.667151 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe71c7-5725-4df1-8447-8b07c60ce472 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:07.679369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 529/1058] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:51:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:07.684239 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dc25e-e0f0-48a5-ae02-ec6c06cbfe7a None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:07.796963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:07.825812 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe71c7-5725-4df1-8447-8b07c60ce472 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:07.840693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dc25e-e0f0-48a5-ae02-ec6c06cbfe7a None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:07.882646 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:07.891090 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832f228-8633-484a-a4cc-59168584058a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:07.891976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832f228-8633-484a-a4cc-59168584058a None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:07.892255 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832f228-8633-484a-a4cc-59168584058a None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:07.892533 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832f228-8633-484a-a4cc-59168584058a None admin] PATH_INFO: `/v3/users/a8b34894f74e43e9b9edd5ad71eef70e` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:07.893091 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5304e-39aa-449f-8cda-235bc8ee85a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:07.893707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5304e-39aa-449f-8cda-235bc8ee85a6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:07.893991 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5304e-39aa-449f-8cda-235bc8ee85a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:07.894276 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5304e-39aa-449f-8cda-235bc8ee85a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:07.895265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5304e-39aa-449f-8cda-235bc8ee85a6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:07.896446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5304e-39aa-449f-8cda-235bc8ee85a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:07.900544 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5304e-39aa-449f-8cda-235bc8ee85a6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:07.908249 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 530/1059] 10.222.0.27 () {56 vars in 1115 bytes} [Sat Jul 16 14:51:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:07.910083 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832f228-8633-484a-a4cc-59168584058a None admin] RBAC: Authorizing `identity:get_user(user_id=a8b34894f74e43e9b9edd5ad71eef70e)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:07.911586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832f228-8633-484a-a4cc-59168584058a None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:07.919105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 530/1060] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:51:07 2022] GET /identity/v3/users/a8b34894f74e43e9b9edd5ad71eef70e => generated 285 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:51:07.919720 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5304e-39aa-449f-8cda-235bc8ee85a6 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:07.919835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832f228-8633-484a-a4cc-59168584058a None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:08.040577 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5304e-39aa-449f-8cda-235bc8ee85a6 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:08.043768 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832f228-8633-484a-a4cc-59168584058a None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:08.088259 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d6f31-f28d-44d2-a3c8-c6b8c3e9a9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:08.091504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d6f31-f28d-44d2-a3c8-c6b8c3e9a9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:08.091504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d6f31-f28d-44d2-a3c8-c6b8c3e9a9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:08.091504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d6f31-f28d-44d2-a3c8-c6b8c3e9a9d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:08.095031 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917a2a5-b341-4c17-a27a-d88dafcd21d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:08.095699 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917a2a5-b341-4c17-a27a-d88dafcd21d2 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:08.095973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917a2a5-b341-4c17-a27a-d88dafcd21d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:08.096249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917a2a5-b341-4c17-a27a-d88dafcd21d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:08.099269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:08.107200 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d6f31-f28d-44d2-a3c8-c6b8c3e9a9d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:08.108828 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d6f31-f28d-44d2-a3c8-c6b8c3e9a9d3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:08.129435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-9e7d6f31-f28d-44d2-a3c8-c6b8c3e9a9d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:51:08.129435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:08.129435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:08.129435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:08.129435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:08.129435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:08.129435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:08.129435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:08.129435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:08.129435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:08.129435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:08.129435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:08.129435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:51:08.129435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:51:08.129435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:51:08.130620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:51:08.130620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:08.130620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:08.130620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:08.130620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:08.130620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:08.130620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:08.130620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:08.130620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:08.130620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:08.130620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:08.130620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:08.130620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:08.130620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:08.130620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:08.130620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:08.130620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:08.131765 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:51:08.131765 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:51:08.131765 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:51:08.131765 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:51:08.131765 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 531/1061] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:51:08.140243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d6f31-f28d-44d2-a3c8-c6b8c3e9a9d3 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:08.168106 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:08.191160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917a2a5-b341-4c17-a27a-d88dafcd21d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:08.193984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917a2a5-b341-4c17-a27a-d88dafcd21d2 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:08.274696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d6f31-f28d-44d2-a3c8-c6b8c3e9a9d3 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:08.317333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb00090-1c6f-438a-ac92-6fb0dfe4d147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:08.318268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb00090-1c6f-438a-ac92-6fb0dfe4d147 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:08.318616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb00090-1c6f-438a-ac92-6fb0dfe4d147 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:08.319600 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb00090-1c6f-438a-ac92-6fb0dfe4d147 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:08.319720 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb00090-1c6f-438a-ac92-6fb0dfe4d147 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:08.319972 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb00090-1c6f-438a-ac92-6fb0dfe4d147 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:08.321389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb00090-1c6f-438a-ac92-6fb0dfe4d147 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:08.333928 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 532/1062] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:08.341705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb00090-1c6f-438a-ac92-6fb0dfe4d147 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:08.355955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 531/1063] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:51:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 16 14:51:08.362977 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917a2a5-b341-4c17-a27a-d88dafcd21d2 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:08.404922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:08.468309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb00090-1c6f-438a-ac92-6fb0dfe4d147 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:08.491728 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:08.514334 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917a2a5-b341-4c17-a27a-d88dafcd21d2 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:08.525794 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8a9ec-df0a-467d-baca-3f37e65ef6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:08.527299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8a9ec-df0a-467d-baca-3f37e65ef6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:08.527299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8a9ec-df0a-467d-baca-3f37e65ef6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:08.527299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8a9ec-df0a-467d-baca-3f37e65ef6f9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:08.538137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8a9ec-df0a-467d-baca-3f37e65ef6f9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:08.540287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8a9ec-df0a-467d-baca-3f37e65ef6f9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:08.559446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-53b8a9ec-df0a-467d-baca-3f37e65ef6f9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 16 14:51:08.559446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:08.559446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:08.559446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:08.559446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:08.559446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:08.559446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:08.559446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:08.559446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:08.559446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:08.559446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:08.559446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:08.559446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:51:08.559446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:51:08.559446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:51:08.559446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:51:08.560611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:08.560611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:08.560611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:51:08.560611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:51:08.560611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:51:08.560611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:51:08.560611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:08.560611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:08.560611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:08.560611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:08.560611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:08.560611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:08.560611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:08.560611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:08.560611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:08.560611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:08.560611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:08.560611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:08.566663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:08.566663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:08.566663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:51:08.566663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:51:08.566663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:51:08.566663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:51:08.566663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:51:08.566663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:51:08.566663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 16 14:51:08.566663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:51:08.566663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 533/1064] 10.222.0.27 () {56 vars in 1093 bytes} [Sat Jul 16 14:51:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:51:08.567341 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8a9ec-df0a-467d-baca-3f37e65ef6f9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:08.578388 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f92a25-3061-4eba-8a20-d0dc8e9d6a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:08.579395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f92a25-3061-4eba-8a20-d0dc8e9d6a8f None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:08.579740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f92a25-3061-4eba-8a20-d0dc8e9d6a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:08.580098 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f92a25-3061-4eba-8a20-d0dc8e9d6a8f None admin] PATH_INFO: `/v3/projects/a9ee6e48dec5416cbf54e8f87992aba8` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:08.581446 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f92a25-3061-4eba-8a20-d0dc8e9d6a8f None admin] RBAC: Authorizing `identity:get_project(project_id=a9ee6e48dec5416cbf54e8f87992aba8)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:08.583031 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f92a25-3061-4eba-8a20-d0dc8e9d6a8f None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:08.585717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 532/1065] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:51:08 2022] GET /identity/v3/projects/a9ee6e48dec5416cbf54e8f87992aba8 => generated 311 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:08.591897 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f92a25-3061-4eba-8a20-d0dc8e9d6a8f None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:08.707073 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8a9ec-df0a-467d-baca-3f37e65ef6f9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:08.721070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:08.739450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f92a25-3061-4eba-8a20-d0dc8e9d6a8f None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:08.770104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe34d57-27fb-48bd-b556-4cd5b6157958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:08.771683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe34d57-27fb-48bd-b556-4cd5b6157958 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:08.776855 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe34d57-27fb-48bd-b556-4cd5b6157958 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:08.776855 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe34d57-27fb-48bd-b556-4cd5b6157958 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:08.776855 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe34d57-27fb-48bd-b556-4cd5b6157958 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:08.777480 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe34d57-27fb-48bd-b556-4cd5b6157958 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:08.778913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe34d57-27fb-48bd-b556-4cd5b6157958 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:08.782073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eed3a0-16ba-4a8f-a531-736c0ce3809e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:08.783596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eed3a0-16ba-4a8f-a531-736c0ce3809e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:08.783982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eed3a0-16ba-4a8f-a531-736c0ce3809e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:08.784257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eed3a0-16ba-4a8f-a531-736c0ce3809e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:08.785256 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eed3a0-16ba-4a8f-a531-736c0ce3809e None admin] RBAC: Adding query filter params (role.id=0efeff46573247f3b9f1e2dec2968672, scope.project.id=a9ee6e48dec5416cbf54e8f87992aba8, user.id=a8b34894f74e43e9b9edd5ad71eef70e) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:08.785649 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eed3a0-16ba-4a8f-a531-736c0ce3809e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:08.788179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eed3a0-16ba-4a8f-a531-736c0ce3809e None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:08.796329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 533/1066] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:51:08 2022] GET /identity/v3/role_assignments?user.id=a8b34894f74e43e9b9edd5ad71eef70e&scope.project.id=a9ee6e48dec5416cbf54e8f87992aba8&role.id=0efeff46573247f3b9f1e2dec2968672 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:08.813842 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 534/1067] 10.222.0.27 () {56 vars in 1133 bytes} [Sat Jul 16 14:51:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:08.815827 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eed3a0-16ba-4a8f-a531-736c0ce3809e None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:08.844787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:08.977548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eed3a0-16ba-4a8f-a531-736c0ce3809e None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:09.018437 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c525b-9c18-453e-8b8d-2cfa88fe8193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:09.019257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c525b-9c18-453e-8b8d-2cfa88fe8193 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:09.019531 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c525b-9c18-453e-8b8d-2cfa88fe8193 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:09.019797 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c525b-9c18-453e-8b8d-2cfa88fe8193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:09.031788 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c525b-9c18-453e-8b8d-2cfa88fe8193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:09.033250 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c525b-9c18-453e-8b8d-2cfa88fe8193 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:09.052740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-5d8c525b-9c18-453e-8b8d-2cfa88fe8193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:51:09.052740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:09.052740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:09.052740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:09.052740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:09.052740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:09.052740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:09.052740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:09.052740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:09.052740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:09.052740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:09.052740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:09.052740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:51:09.052740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:51:09.052740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:51:09.054000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:51:09.054000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:09.054000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:09.054000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:09.054000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:09.054000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:09.054000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:09.054000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:09.054000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:09.054000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:09.054000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:09.054000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:09.054000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:09.054000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:09.054000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:09.054000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:09.054000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:09.060348 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:51:09.060348 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:51:09.060348 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:51:09.060348 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:51:09.060348 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 534/1068] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:51:09.062227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe34d57-27fb-48bd-b556-4cd5b6157958 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:09.125582 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972aa3ea-a1ca-4e7b-ac6c-7e0ac621e026 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:09.125910 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972aa3ea-a1ca-4e7b-ac6c-7e0ac621e026 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:09.126141 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972aa3ea-a1ca-4e7b-ac6c-7e0ac621e026 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:09.127769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 535/1069] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:51:09.133480 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d5407-0bb4-4017-aa56-2540aa10b1a4 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:09.133832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d5407-0bb4-4017-aa56-2540aa10b1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:09.134116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d5407-0bb4-4017-aa56-2540aa10b1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:09.151514 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-f91d5407-0bb4-4017-aa56-2540aa10b1a4 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:09.152720 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-f91d5407-0bb4-4017-aa56-2540aa10b1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:09.195237 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe34d57-27fb-48bd-b556-4cd5b6157958 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:09.230406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:09.235731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 536/1070] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:51:09.236017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f21c04-7c8e-4fc7-9e2f-c7b325035947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:09.236329 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f21c04-7c8e-4fc7-9e2f-c7b325035947 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:09.236602 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f21c04-7c8e-4fc7-9e2f-c7b325035947 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:09.236863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f21c04-7c8e-4fc7-9e2f-c7b325035947 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:09.237598 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f21c04-7c8e-4fc7-9e2f-c7b325035947 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:09.237909 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f21c04-7c8e-4fc7-9e2f-c7b325035947 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:09.239354 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f21c04-7c8e-4fc7-9e2f-c7b325035947 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:09.253703 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 535/1071] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:09.260146 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f21c04-7c8e-4fc7-9e2f-c7b325035947 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:09.260334 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d5407-0bb4-4017-aa56-2540aa10b1a4 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:09.295465 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:09.393920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f21c04-7c8e-4fc7-9e2f-c7b325035947 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:09.394294 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d5407-0bb4-4017-aa56-2540aa10b1a4 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:09.434393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7941f52-efaa-4f55-b9ec-2a329c3bb67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:09.434966 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeca676-cbbf-43d2-844a-9dd43d16ffab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:09.435359 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7941f52-efaa-4f55-b9ec-2a329c3bb67e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:09.435624 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeca676-cbbf-43d2-844a-9dd43d16ffab None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:09.435725 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7941f52-efaa-4f55-b9ec-2a329c3bb67e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:09.435906 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeca676-cbbf-43d2-844a-9dd43d16ffab None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:09.436003 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7941f52-efaa-4f55-b9ec-2a329c3bb67e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:09.436175 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeca676-cbbf-43d2-844a-9dd43d16ffab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:09.444880 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7941f52-efaa-4f55-b9ec-2a329c3bb67e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:09.446980 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7941f52-efaa-4f55-b9ec-2a329c3bb67e None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:09.463230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-a7941f52-efaa-4f55-b9ec-2a329c3bb67e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:51:09.463230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:09.463230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:09.463230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:09.463230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:09.463230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:09.463230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:09.463230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:09.463230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:09.463230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:09.463230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:09.463230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:09.463230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:51:09.463230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:51:09.463230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:51:09.464208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:51:09.464208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:09.464208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:09.464208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:09.464208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:09.464208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:09.464208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:09.464208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:09.464208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:09.464208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:09.464208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:09.464208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:09.464208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:09.464208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:09.464208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:09.464208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:09.464208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:09.465199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:51:09.465199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:51:09.465199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:51:09.465199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:51:09.465199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:51:09.465199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:51:09.465199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:51:09.465199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:51:09.465199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 537/1072] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:51:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:51:09.470268 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7941f52-efaa-4f55-b9ec-2a329c3bb67e None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:09.516826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeca676-cbbf-43d2-844a-9dd43d16ffab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:09.518590 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeca676-cbbf-43d2-844a-9dd43d16ffab None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:09.536302 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:09.550190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:09.581536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7941f52-efaa-4f55-b9ec-2a329c3bb67e None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:09.619600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bb0c4-f92f-415b-98bd-c65709144a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:09.620602 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bb0c4-f92f-415b-98bd-c65709144a60 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:09.621078 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bb0c4-f92f-415b-98bd-c65709144a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:09.621542 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bb0c4-f92f-415b-98bd-c65709144a60 None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:09.622680 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bb0c4-f92f-415b-98bd-c65709144a60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:09.623212 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bb0c4-f92f-415b-98bd-c65709144a60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:09.624819 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bb0c4-f92f-415b-98bd-c65709144a60 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:09.627406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 536/1073] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:51:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 16 14:51:09.634220 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeca676-cbbf-43d2-844a-9dd43d16ffab None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:09.643538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 538/1074] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:51:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:51:09.650009 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bb0c4-f92f-415b-98bd-c65709144a60 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:09.748231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeca676-cbbf-43d2-844a-9dd43d16ffab None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:09.754796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bb0c4-f92f-415b-98bd-c65709144a60 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:09.777813 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:09.784302 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:09.797716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735cf4b-a660-43fa-a6b6-0ff3f0914e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:09.798698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735cf4b-a660-43fa-a6b6-0ff3f0914e7b None admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:09.798812 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6fa21-10ef-45f5-9d0d-cde27e50decf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:09.799066 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735cf4b-a660-43fa-a6b6-0ff3f0914e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:09.799300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735cf4b-a660-43fa-a6b6-0ff3f0914e7b None admin] PATH_INFO: `/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/3f2e241494244edf8d283e8f44192c46/roles/e3c9141bd01741b59c8701275be9154c` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:09.799552 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6fa21-10ef-45f5-9d0d-cde27e50decf None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:09.799881 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6fa21-10ef-45f5-9d0d-cde27e50decf None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:09.800136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6fa21-10ef-45f5-9d0d-cde27e50decf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:09.802065 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735cf4b-a660-43fa-a6b6-0ff3f0914e7b None admin] RBAC: Authorizing `identity:create_grant(project_id=2739f00eb01e46a9acf90ffa360d7d8c, user_id=3f2e241494244edf8d283e8f44192c46, role_id=e3c9141bd01741b59c8701275be9154c)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:09.804058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735cf4b-a660-43fa-a6b6-0ff3f0914e7b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:09.812531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6fa21-10ef-45f5-9d0d-cde27e50decf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:09.813993 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6fa21-10ef-45f5-9d0d-cde27e50decf None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:09.832164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-74e6fa21-10ef-45f5-9d0d-cde27e50decf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:51:09.832164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:09.832164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:09.832164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:09.832164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:09.832164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:09.832164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:09.832164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:09.832164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:09.832164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:09.832164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:09.832164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:09.832164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:51:09.832164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:51:09.832164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:51:09.833389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:51:09.833389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:09.833389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:09.833389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:09.833389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:09.833389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:09.833389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:09.833389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:09.833389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:09.833389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:09.833389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:09.833389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:09.833389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:09.833389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:09.833389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:09.833389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:09.833389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:09.835027 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:51:09.835027 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:51:09.835027 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:51:09.835027 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:51:09.835027 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 537/1075] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:51:09.840634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6fa21-10ef-45f5-9d0d-cde27e50decf None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:09.925215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 539/1076] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:51:09 2022] PUT /identity/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/3f2e241494244edf8d283e8f44192c46/roles/e3c9141bd01741b59c8701275be9154c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 14:51:09.951646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6fa21-10ef-45f5-9d0d-cde27e50decf None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:09.993427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0787fcc-0fd2-48e0-b30f-3d2479b0eeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:09.994121 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0787fcc-0fd2-48e0-b30f-3d2479b0eeef None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:09.994403 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0787fcc-0fd2-48e0-b30f-3d2479b0eeef None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:09.994676 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0787fcc-0fd2-48e0-b30f-3d2479b0eeef None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:09.995454 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0787fcc-0fd2-48e0-b30f-3d2479b0eeef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:09.995762 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0787fcc-0fd2-48e0-b30f-3d2479b0eeef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:09.997239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0787fcc-0fd2-48e0-b30f-3d2479b0eeef None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:10.015192 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 538/1077] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:10.854755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb546c-d04e-4496-8a70-ee59814a032a None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:10.855355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb546c-d04e-4496-8a70-ee59814a032a None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:10.855355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb546c-d04e-4496-8a70-ee59814a032a None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:10.856703 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 540/1078] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:10.862027 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4ff02-5c1f-4426-afb7-a09987497e14 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:10.862507 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4ff02-5c1f-4426-afb7-a09987497e14 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:10.862605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4ff02-5c1f-4426-afb7-a09987497e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:10.880375 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f6a4ff02-5c1f-4426-afb7-a09987497e14 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:10.882231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f6a4ff02-5c1f-4426-afb7-a09987497e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:10.939629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:10.977724 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 539/1079] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:51:11.000617 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb546c-d04e-4496-8a70-ee59814a032a None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:11.120732 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb546c-d04e-4496-8a70-ee59814a032a None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:11.157154 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac4418-bde0-4c8a-8c83-fe3d0066a112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:11.157939 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac4418-bde0-4c8a-8c83-fe3d0066a112 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:11.158215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac4418-bde0-4c8a-8c83-fe3d0066a112 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:11.158490 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac4418-bde0-4c8a-8c83-fe3d0066a112 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:11.169920 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac4418-bde0-4c8a-8c83-fe3d0066a112 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:51:11.177495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-e0ac4418-bde0-4c8a-8c83-fe3d0066a112 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:51:11.177495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:11.177495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:11.177495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:11.177495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:11.177495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:11.177495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:11.177495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:11.177495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:11.177495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:11.177495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:11.177495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:11.177495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:51:11.177495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:51:11.177495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:51:11.177495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:51:11.179415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:51:11.179415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:51:11.179415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:51:11.179415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:51:11.179415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:11.179415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:11.179415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:11.179415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:11.179415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:11.179415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:11.179415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:11.179415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:11.179415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:11.179415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:11.179415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:11.179415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:11.180282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:11.180282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:11.180282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:11.180282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:11.180282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:51:11.180282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:51:11.180282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:51:11.180282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:51:11.180282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:51:11.180282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:51:11.180282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:51:11.180282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:51:11.180282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 541/1080] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:51:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:51:11.186106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4ff02-5c1f-4426-afb7-a09987497e14 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:11.293078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4ff02-5c1f-4426-afb7-a09987497e14 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:11.333222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03dd74e-5e54-4d5f-834a-693f2d6fe3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:11.334811 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dd74e-5e54-4d5f-834a-693f2d6fe3dd None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:11.335118 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dd74e-5e54-4d5f-834a-693f2d6fe3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:11.335393 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dd74e-5e54-4d5f-834a-693f2d6fe3dd None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:11.336358 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03dd74e-5e54-4d5f-834a-693f2d6fe3dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:11.336915 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03dd74e-5e54-4d5f-834a-693f2d6fe3dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:11.338303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03dd74e-5e54-4d5f-834a-693f2d6fe3dd None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:11.347028 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 540/1081] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:51:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:11.354194 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac4418-bde0-4c8a-8c83-fe3d0066a112 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:11.405094 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:11.484009 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac4418-bde0-4c8a-8c83-fe3d0066a112 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:11.537413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed14d1-074f-42f1-9cb5-b5a83e4e4908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:11.538773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed14d1-074f-42f1-9cb5-b5a83e4e4908 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:11.539329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed14d1-074f-42f1-9cb5-b5a83e4e4908 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:11.539839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed14d1-074f-42f1-9cb5-b5a83e4e4908 None admin] PATH_INFO: `/v3/users/a8b34894f74e43e9b9edd5ad71eef70e` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:11.541953 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afed14d1-074f-42f1-9cb5-b5a83e4e4908 None admin] RBAC: Authorizing `identity:get_user(user_id=a8b34894f74e43e9b9edd5ad71eef70e)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:11.544626 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afed14d1-074f-42f1-9cb5-b5a83e4e4908 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:11.547836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 542/1082] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:51:11 2022] GET /identity/v3/users/a8b34894f74e43e9b9edd5ad71eef70e => generated 285 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:11.555645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03dd74e-5e54-4d5f-834a-693f2d6fe3dd None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:11.610252 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:11.685247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03dd74e-5e54-4d5f-834a-693f2d6fe3dd None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:11.728149 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae0bfd-4c5f-4648-a764-dac525fda1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:11.728930 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae0bfd-4c5f-4648-a764-dac525fda1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:11.729208 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae0bfd-4c5f-4648-a764-dac525fda1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:11.729474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae0bfd-4c5f-4648-a764-dac525fda1e7 None admin] PATH_INFO: `/v3/projects/a9ee6e48dec5416cbf54e8f87992aba8` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:11.730689 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae0bfd-4c5f-4648-a764-dac525fda1e7 None admin] RBAC: Authorizing `identity:get_project(project_id=a9ee6e48dec5416cbf54e8f87992aba8)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:11.732171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae0bfd-4c5f-4648-a764-dac525fda1e7 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:11.733911 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 541/1083] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:51:11 2022] GET /identity/v3/projects/a9ee6e48dec5416cbf54e8f87992aba8 => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:11.742307 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed14d1-074f-42f1-9cb5-b5a83e4e4908 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:11.862316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed14d1-074f-42f1-9cb5-b5a83e4e4908 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:11.873799 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:11.904325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fb77b-fd86-412a-92d5-e4c313a4d99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:11.905243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fb77b-fd86-412a-92d5-e4c313a4d99a None admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:11.905529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fb77b-fd86-412a-92d5-e4c313a4d99a None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:11.905811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fb77b-fd86-412a-92d5-e4c313a4d99a None admin] PATH_INFO: `/v3/projects/a9ee6e48dec5416cbf54e8f87992aba8/users/a8b34894f74e43e9b9edd5ad71eef70e/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:11.907953 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9fb77b-fd86-412a-92d5-e4c313a4d99a None admin] RBAC: Authorizing `identity:create_grant(project_id=a9ee6e48dec5416cbf54e8f87992aba8, user_id=a8b34894f74e43e9b9edd5ad71eef70e, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:11.909426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9fb77b-fd86-412a-92d5-e4c313a4d99a None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:11.982748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35b18d-aebc-41d1-b26e-7f792b75e627 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:11.983366 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35b18d-aebc-41d1-b26e-7f792b75e627 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:11.983366 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35b18d-aebc-41d1-b26e-7f792b75e627 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:11.984923 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 542/1084] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:11.993808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901671b6-993a-4949-a370-1467fe4e514f None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:11.994353 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901671b6-993a-4949-a370-1467fe4e514f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:11.994796 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901671b6-993a-4949-a370-1467fe4e514f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:11.997399 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 543/1085] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:51:11 2022] PUT /identity/v3/projects/a9ee6e48dec5416cbf54e8f87992aba8/users/a8b34894f74e43e9b9edd5ad71eef70e/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:51:12.021140 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-901671b6-993a-4949-a370-1467fe4e514f None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:12.024018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-901671b6-993a-4949-a370-1467fe4e514f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:12.113128 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 543/1086] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:51:12.138143 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fb77b-fd86-412a-92d5-e4c313a4d99a None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:12.257588 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fb77b-fd86-412a-92d5-e4c313a4d99a None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:12.296068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b664e1-509d-44ae-a06e-27e30ea53895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:12.296885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b664e1-509d-44ae-a06e-27e30ea53895 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:12.297162 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b664e1-509d-44ae-a06e-27e30ea53895 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:12.297435 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b664e1-509d-44ae-a06e-27e30ea53895 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:12.308399 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b664e1-509d-44ae-a06e-27e30ea53895 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:51:12.317467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-54b664e1-509d-44ae-a06e-27e30ea53895 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 14:51:12.317467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:12.317467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:12.317467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:12.317467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:12.317467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:12.317467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:12.317467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:12.317467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:12.317467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:12.317467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:12.317467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:12.317467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:51:12.317467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:51:12.317467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:51:12.317467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:51:12.321921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:51:12.321921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:51:12.321921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:51:12.321921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:51:12.321921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:12.321921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:12.321921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:12.321921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:12.321921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:12.321921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:12.321921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:12.321921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:12.321921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:12.321921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:12.321921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:12.321921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:12.323296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:12.323296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:12.323296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:12.323296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:12.323296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:51:12.323296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:51:12.323296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:51:12.323296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:51:12.323296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:51:12.323296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:51:12.323296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 16 14:51:12.323296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:51:12.323296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 544/1087] 10.222.0.27 () {56 vars in 1120 bytes} [Sat Jul 16 14:51:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:51:12.326118 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901671b6-993a-4949-a370-1467fe4e514f None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:12.378459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:12.454051 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901671b6-993a-4949-a370-1467fe4e514f None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:12.493255 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbbc54-fc45-4cc8-b1d0-6922fb03c29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:12.493980 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbbc54-fc45-4cc8-b1d0-6922fb03c29d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:12.494435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbbc54-fc45-4cc8-b1d0-6922fb03c29d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:12.494725 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbbc54-fc45-4cc8-b1d0-6922fb03c29d None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:12.495511 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbbc54-fc45-4cc8-b1d0-6922fb03c29d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:12.495818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbbc54-fc45-4cc8-b1d0-6922fb03c29d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:12.497201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbbc54-fc45-4cc8-b1d0-6922fb03c29d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:12.507867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 544/1088] 10.222.0.27 () {56 vars in 1115 bytes} [Sat Jul 16 14:51:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:51:12.513829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b664e1-509d-44ae-a06e-27e30ea53895 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:12.544431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:12.626272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b664e1-509d-44ae-a06e-27e30ea53895 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:12.664914 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890239e-efb4-4bcf-98e1-c5f4b1af0352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:12.665817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890239e-efb4-4bcf-98e1-c5f4b1af0352 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:12.666095 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890239e-efb4-4bcf-98e1-c5f4b1af0352 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:12.666691 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890239e-efb4-4bcf-98e1-c5f4b1af0352 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:12.680408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890239e-efb4-4bcf-98e1-c5f4b1af0352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:12.682547 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890239e-efb4-4bcf-98e1-c5f4b1af0352 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:12.701751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-2890239e-efb4-4bcf-98e1-c5f4b1af0352 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:51:12.701751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:12.701751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:12.701751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:12.701751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:12.701751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:12.701751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:12.701751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:12.701751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:12.701751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:12.701751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:12.701751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:12.701751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:51:12.701751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:51:12.701751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:51:12.703291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:51:12.703291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:12.703291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:12.703291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:12.703291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:12.703291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:12.703291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:12.703291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:12.703291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:12.703291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:12.703291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:12.703291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:12.703291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:12.703291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:12.703291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:12.703291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:12.703291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:12.704203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:51:12.704203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:51:12.704203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:51:12.704203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:51:12.704203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 545/1089] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:51:12.708958 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbbc54-fc45-4cc8-b1d0-6922fb03c29d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:12.811640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:12.820899 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbbc54-fc45-4cc8-b1d0-6922fb03c29d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:12.858607 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acb0fe-a3ac-4436-8613-5a0f46b4128f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:12.859301 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acb0fe-a3ac-4436-8613-5a0f46b4128f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:12.859581 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acb0fe-a3ac-4436-8613-5a0f46b4128f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:12.859852 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acb0fe-a3ac-4436-8613-5a0f46b4128f None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:12.860576 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4acb0fe-a3ac-4436-8613-5a0f46b4128f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:12.860878 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4acb0fe-a3ac-4436-8613-5a0f46b4128f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:12.862240 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4acb0fe-a3ac-4436-8613-5a0f46b4128f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:12.876671 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 545/1090] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:12.883347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890239e-efb4-4bcf-98e1-c5f4b1af0352 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:12.974353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:12.996232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890239e-efb4-4bcf-98e1-c5f4b1af0352 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:13.033610 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4ade1-de02-4665-aebf-47dc9e524b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:13.034410 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4ade1-de02-4665-aebf-47dc9e524b06 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:13.034694 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4ade1-de02-4665-aebf-47dc9e524b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:13.035000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4ade1-de02-4665-aebf-47dc9e524b06 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:13.043636 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4ade1-de02-4665-aebf-47dc9e524b06 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:13.045098 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4ade1-de02-4665-aebf-47dc9e524b06 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:13.060167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-6fc4ade1-de02-4665-aebf-47dc9e524b06 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 16 14:51:13.060167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:13.060167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:13.060167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:13.060167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:13.060167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:13.060167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:13.060167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:13.060167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:13.060167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:13.060167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:13.060167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:13.060167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:51:13.060167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:51:13.060167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:51:13.060167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:51:13.066811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:13.066811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:13.066811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:51:13.066811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:51:13.066811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:51:13.066811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:51:13.066811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:13.066811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:13.066811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:13.066811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:13.066811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:13.066811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:13.066811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:13.066811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:13.066811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:13.066811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:13.066811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:13.066811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:13.068082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:13.068082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:13.068082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:51:13.068082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:51:13.068082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:51:13.068082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:51:13.068082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:51:13.068082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:51:13.068082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 16 14:51:13.068082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:51:13.068082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 546/1091] 10.222.0.27 () {56 vars in 1093 bytes} [Sat Jul 16 14:51:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:51:13.068950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acb0fe-a3ac-4436-8613-5a0f46b4128f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:13.185585 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acb0fe-a3ac-4436-8613-5a0f46b4128f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:13.226113 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab0fb1-5b88-45ba-bc34-ebac719fa3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:13.226789 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab0fb1-5b88-45ba-bc34-ebac719fa3ef None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:13.227095 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab0fb1-5b88-45ba-bc34-ebac719fa3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:13.227375 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab0fb1-5b88-45ba-bc34-ebac719fa3ef None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:13.228222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab0fb1-5b88-45ba-bc34-ebac719fa3ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:13.228524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab0fb1-5b88-45ba-bc34-ebac719fa3ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:13.229960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab0fb1-5b88-45ba-bc34-ebac719fa3ef None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:13.247978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 546/1092] 10.222.0.27 () {56 vars in 1133 bytes} [Sat Jul 16 14:51:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:13.254404 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4ade1-de02-4665-aebf-47dc9e524b06 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:13.358321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4ade1-de02-4665-aebf-47dc9e524b06 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:13.393713 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a25bc-cac1-4a76-9620-473f365b0ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:13.394348 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a25bc-cac1-4a76-9620-473f365b0ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:13.394622 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a25bc-cac1-4a76-9620-473f365b0ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:13.395002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a25bc-cac1-4a76-9620-473f365b0ad1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:13.406216 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:13.466747 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a25bc-cac1-4a76-9620-473f365b0ad1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:13.468293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835a25bc-cac1-4a76-9620-473f365b0ad1 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:13.573445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 547/1093] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:51:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 14:51:13.579270 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab0fb1-5b88-45ba-bc34-ebac719fa3ef None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:13.594032 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:13.688372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab0fb1-5b88-45ba-bc34-ebac719fa3ef None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:13.724304 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361699b3-fd7a-4b5d-935a-c0e6eb07a3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:13.725265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361699b3-fd7a-4b5d-935a-c0e6eb07a3da None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:13.725631 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361699b3-fd7a-4b5d-935a-c0e6eb07a3da None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:13.725995 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361699b3-fd7a-4b5d-935a-c0e6eb07a3da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:13.737826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361699b3-fd7a-4b5d-935a-c0e6eb07a3da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:13.739357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361699b3-fd7a-4b5d-935a-c0e6eb07a3da None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:13.760538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-361699b3-fd7a-4b5d-935a-c0e6eb07a3da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:51:13.760538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:13.760538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:13.760538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:13.760538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:13.760538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:13.760538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:13.760538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:13.760538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:13.760538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:13.760538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:13.760538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:13.760538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:51:13.760538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:51:13.760538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:51:13.767491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:51:13.767491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:13.767491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:13.767491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:13.767491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:13.767491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:13.767491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:13.767491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:13.767491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:13.767491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:13.767491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:13.767491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:13.767491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:13.767491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:13.767491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:13.767491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:13.767491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:13.768412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:51:13.768412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:51:13.768412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:51:13.768412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:51:13.768412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 547/1094] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:51:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:51:13.768705 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a25bc-cac1-4a76-9620-473f365b0ad1 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:13.850537 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:13.926017 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a25bc-cac1-4a76-9620-473f365b0ad1 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:13.967903 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51e459-0b52-4e57-8e7a-eb22aa921e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:13.968496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51e459-0b52-4e57-8e7a-eb22aa921e1d None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:13.968771 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51e459-0b52-4e57-8e7a-eb22aa921e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:13.969043 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51e459-0b52-4e57-8e7a-eb22aa921e1d None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:13.969772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51e459-0b52-4e57-8e7a-eb22aa921e1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:13.970071 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51e459-0b52-4e57-8e7a-eb22aa921e1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:13.971480 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51e459-0b52-4e57-8e7a-eb22aa921e1d None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:13.984656 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 548/1095] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:13.990169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361699b3-fd7a-4b5d-935a-c0e6eb07a3da None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:14.058127 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:14.098433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361699b3-fd7a-4b5d-935a-c0e6eb07a3da None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:14.103525 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f7a9d-2375-4b3f-99f4-09cedab4824d None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:14.103816 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f7a9d-2375-4b3f-99f4-09cedab4824d None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:14.104094 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f7a9d-2375-4b3f-99f4-09cedab4824d None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:14.105497 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 549/1096] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:51:14.112166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f1cc6-8da7-491d-8110-83c228fb9009 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:14.112454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f1cc6-8da7-491d-8110-83c228fb9009 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:14.112726 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f1cc6-8da7-491d-8110-83c228fb9009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:14.130647 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-7f7f1cc6-8da7-491d-8110-83c228fb9009 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:14.132340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7f7f1cc6-8da7-491d-8110-83c228fb9009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:14.139422 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48a9a8-1eb9-4c4f-a476-9d2ef450616f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:14.140229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48a9a8-1eb9-4c4f-a476-9d2ef450616f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:14.141024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48a9a8-1eb9-4c4f-a476-9d2ef450616f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:14.141306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48a9a8-1eb9-4c4f-a476-9d2ef450616f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:14.149589 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48a9a8-1eb9-4c4f-a476-9d2ef450616f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:14.151089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48a9a8-1eb9-4c4f-a476-9d2ef450616f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:14.165840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-0b48a9a8-1eb9-4c4f-a476-9d2ef450616f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:51:14.165840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:14.165840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:14.165840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:14.165840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:14.165840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:14.165840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:14.165840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:14.165840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:14.165840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:14.165840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:14.165840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:14.165840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:51:14.165840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:51:14.165840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:51:14.166913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:51:14.166913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:14.166913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:14.166913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:14.166913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:14.166913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:14.166913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:14.166913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:14.166913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:14.166913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:14.166913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:14.166913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:14.166913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:14.166913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:14.166913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:14.166913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:14.166913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:14.167844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:51:14.167844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:51:14.167844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:51:14.167844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:51:14.167844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:51:14.167844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:51:14.167844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:51:14.167844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:51:14.167844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 548/1097] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:51:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:51:14.173259 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48a9a8-1eb9-4c4f-a476-9d2ef450616f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:14.213575 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 550/1098] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:51:14.238373 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f1cc6-8da7-491d-8110-83c228fb9009 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:14.270430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:14.285466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48a9a8-1eb9-4c4f-a476-9d2ef450616f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:14.313471 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:14.328687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6d958-b9b0-4dc4-91a5-33cd3dc2034c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:14.329370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6d958-b9b0-4dc4-91a5-33cd3dc2034c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:14.329653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6d958-b9b0-4dc4-91a5-33cd3dc2034c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:14.329930 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6d958-b9b0-4dc4-91a5-33cd3dc2034c None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:14.330738 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6d958-b9b0-4dc4-91a5-33cd3dc2034c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:14.331093 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6d958-b9b0-4dc4-91a5-33cd3dc2034c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:14.332550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6d958-b9b0-4dc4-91a5-33cd3dc2034c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:14.352814 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 549/1099] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:51:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:14.354355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f1cc6-8da7-491d-8110-83c228fb9009 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:14.354663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6d958-b9b0-4dc4-91a5-33cd3dc2034c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:14.390131 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a70aa-f280-476a-9108-87ce8ad531a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:14.390957 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a70aa-f280-476a-9108-87ce8ad531a5 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:14.391230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a70aa-f280-476a-9108-87ce8ad531a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:14.391502 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a70aa-f280-476a-9108-87ce8ad531a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:14.403299 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a70aa-f280-476a-9108-87ce8ad531a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:51:14.410952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-ae0a70aa-f280-476a-9108-87ce8ad531a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:51:14.410952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:14.410952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:14.410952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:14.410952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:14.410952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:14.410952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:14.410952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:14.410952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:14.410952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:14.410952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:14.410952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:14.410952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:51:14.410952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:51:14.410952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:51:14.410952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:51:14.412120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:51:14.412120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:51:14.412120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:51:14.412120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:51:14.412120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:14.412120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:14.412120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:14.412120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:14.412120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:14.412120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:14.412120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:14.412120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:14.412120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:14.412120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:14.412120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:14.412120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:14.413579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:14.413579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:14.413579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:14.413579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:14.413579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:51:14.413579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:51:14.413579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:51:14.413579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:51:14.413579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:51:14.413579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:51:14.413579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:51:14.413579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:51:14.413579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 551/1100] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:51:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:51:14.419227 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a70aa-f280-476a-9108-87ce8ad531a5 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:14.497016 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6d958-b9b0-4dc4-91a5-33cd3dc2034c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:14.528996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:14.538444 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc1459-edf7-4723-8ad7-17ff95079958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:14.539356 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc1459-edf7-4723-8ad7-17ff95079958 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:14.539686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc1459-edf7-4723-8ad7-17ff95079958 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:14.539974 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc1459-edf7-4723-8ad7-17ff95079958 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:14.541023 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc1459-edf7-4723-8ad7-17ff95079958 None admin] RBAC: Adding query filter params (role.id=e3c9141bd01741b59c8701275be9154c, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=3f2e241494244edf8d283e8f44192c46) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:14.545031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc1459-edf7-4723-8ad7-17ff95079958 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:14.545208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a70aa-f280-476a-9108-87ce8ad531a5 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:14.545331 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc1459-edf7-4723-8ad7-17ff95079958 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:14.552592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 550/1101] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:51:14 2022] GET /identity/v3/role_assignments?user.id=3f2e241494244edf8d283e8f44192c46&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=e3c9141bd01741b59c8701275be9154c => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:14.583275 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86376443-2b5a-4c02-9252-08480c85acb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:14.583968 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86376443-2b5a-4c02-9252-08480c85acb0 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:14.584807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86376443-2b5a-4c02-9252-08480c85acb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:14.585877 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86376443-2b5a-4c02-9252-08480c85acb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:14.586947 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86376443-2b5a-4c02-9252-08480c85acb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:14.587260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86376443-2b5a-4c02-9252-08480c85acb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:14.588699 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86376443-2b5a-4c02-9252-08480c85acb0 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:14.598601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 552/1102] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:51:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:14.605370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc1459-edf7-4723-8ad7-17ff95079958 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:14.670583 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:14.741478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc1459-edf7-4723-8ad7-17ff95079958 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:14.780440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3eaf3-16bd-473b-be9c-0f5df22b14fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:14.781254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3eaf3-16bd-473b-be9c-0f5df22b14fb None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:14.781536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3eaf3-16bd-473b-be9c-0f5df22b14fb None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:14.781808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3eaf3-16bd-473b-be9c-0f5df22b14fb None admin] PATH_INFO: `/v3/users/a8b34894f74e43e9b9edd5ad71eef70e` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:14.783069 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3eaf3-16bd-473b-be9c-0f5df22b14fb None admin] RBAC: Authorizing `identity:get_user(user_id=a8b34894f74e43e9b9edd5ad71eef70e)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:14.784553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3eaf3-16bd-473b-be9c-0f5df22b14fb None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:14.786207 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 551/1103] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:51:14 2022] GET /identity/v3/users/a8b34894f74e43e9b9edd5ad71eef70e => generated 285 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:14.793006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86376443-2b5a-4c02-9252-08480c85acb0 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:14.907735 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86376443-2b5a-4c02-9252-08480c85acb0 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:14.947302 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9c18c-38cb-4ffd-9d48-a33fb9c51dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:14.947973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9c18c-38cb-4ffd-9d48-a33fb9c51dfb None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:14.948257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9c18c-38cb-4ffd-9d48-a33fb9c51dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:14.948588 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9c18c-38cb-4ffd-9d48-a33fb9c51dfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:14.973269 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:15.023826 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c9c18c-38cb-4ffd-9d48-a33fb9c51dfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:15.025293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c9c18c-38cb-4ffd-9d48-a33fb9c51dfb None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:15.132274 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 553/1104] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:51:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 14:51:15.138317 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3eaf3-16bd-473b-be9c-0f5df22b14fb None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:15.262616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3eaf3-16bd-473b-be9c-0f5df22b14fb None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:15.278264 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:15.300901 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60899c88-0bfa-463a-ba30-a4820815a419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:15.301790 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60899c88-0bfa-463a-ba30-a4820815a419 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:15.302061 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60899c88-0bfa-463a-ba30-a4820815a419 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:15.302334 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60899c88-0bfa-463a-ba30-a4820815a419 None admin] PATH_INFO: `/v3/projects/a9ee6e48dec5416cbf54e8f87992aba8` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:15.304214 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60899c88-0bfa-463a-ba30-a4820815a419 None admin] RBAC: Authorizing `identity:get_project(project_id=a9ee6e48dec5416cbf54e8f87992aba8)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:15.305800 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60899c88-0bfa-463a-ba30-a4820815a419 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:15.307641 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 552/1105] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:51:15 2022] GET /identity/v3/projects/a9ee6e48dec5416cbf54e8f87992aba8 => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:15.313938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9c18c-38cb-4ffd-9d48-a33fb9c51dfb None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:15.380047 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:15.436988 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9c18c-38cb-4ffd-9d48-a33fb9c51dfb None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:15.478814 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06728a30-c352-45ad-8781-3561dac3ba39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:15.479505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06728a30-c352-45ad-8781-3561dac3ba39 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:15.479783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06728a30-c352-45ad-8781-3561dac3ba39 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:15.480059 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06728a30-c352-45ad-8781-3561dac3ba39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:15.480905 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06728a30-c352-45ad-8781-3561dac3ba39 None admin] RBAC: Adding query filter params (role.id=0efeff46573247f3b9f1e2dec2968672, scope.project.id=a9ee6e48dec5416cbf54e8f87992aba8, user.id=a8b34894f74e43e9b9edd5ad71eef70e) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:15.481213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06728a30-c352-45ad-8781-3561dac3ba39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:15.482664 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06728a30-c352-45ad-8781-3561dac3ba39 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:15.489147 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 554/1106] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:51:15 2022] GET /identity/v3/role_assignments?user.id=a8b34894f74e43e9b9edd5ad71eef70e&scope.project.id=a9ee6e48dec5416cbf54e8f87992aba8&role.id=0efeff46573247f3b9f1e2dec2968672 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:17.562448 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a60acc-74f9-4950-b307-799d763506ff None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:17.563032 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a60acc-74f9-4950-b307-799d763506ff None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:17.563032 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a60acc-74f9-4950-b307-799d763506ff None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:17.564392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 553/1107] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:17.569722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc87e1-2e29-4031-b2b4-2c5cf73d2ac2 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:17.570162 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc87e1-2e29-4031-b2b4-2c5cf73d2ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:17.570266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc87e1-2e29-4031-b2b4-2c5cf73d2ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:17.587879 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-b8bc87e1-2e29-4031-b2b4-2c5cf73d2ac2 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:17.589073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b8bc87e1-2e29-4031-b2b4-2c5cf73d2ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:17.655066 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 555/1108] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:51:17.679809 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a60acc-74f9-4950-b307-799d763506ff None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:17.783555 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a60acc-74f9-4950-b307-799d763506ff None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:17.918679 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4ff31-f387-45b6-bf5e-7858ce9aa664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:17.919527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4ff31-f387-45b6-bf5e-7858ce9aa664 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:17.919805 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4ff31-f387-45b6-bf5e-7858ce9aa664 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:17.920075 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4ff31-f387-45b6-bf5e-7858ce9aa664 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:17.921278 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4ff31-f387-45b6-bf5e-7858ce9aa664 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:17.922720 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4ff31-f387-45b6-bf5e-7858ce9aa664 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:17.924464 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 554/1109] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:51:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:17.930669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc87e1-2e29-4031-b2b4-2c5cf73d2ac2 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:17.979414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:18.033027 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc87e1-2e29-4031-b2b4-2c5cf73d2ac2 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:18.069043 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ffe47-94a5-49b6-9236-8d6932885543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:18.069853 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ffe47-94a5-49b6-9236-8d6932885543 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:18.070188 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ffe47-94a5-49b6-9236-8d6932885543 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:18.070470 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ffe47-94a5-49b6-9236-8d6932885543 None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:18.071219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ffe47-94a5-49b6-9236-8d6932885543 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:18.072665 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ffe47-94a5-49b6-9236-8d6932885543 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:18.226922 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 556/1110] 10.222.0.27 () {60 vars in 1132 bytes} [Sat Jul 16 14:51:17 2022] POST /identity/v3/users => generated 286 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:51:20.302551 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515363fb-b5ec-46cf-bed5-10588549b08c None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:20.303114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515363fb-b5ec-46cf-bed5-10588549b08c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:20.303114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515363fb-b5ec-46cf-bed5-10588549b08c None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:20.304434 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 555/1111] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:20.309984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a1239-1850-41ae-9da1-ea9f60c8fd2b None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:20.318988 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a1239-1850-41ae-9da1-ea9f60c8fd2b None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:20.318988 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a1239-1850-41ae-9da1-ea9f60c8fd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:20.328462 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-262a1239-1850-41ae-9da1-ea9f60c8fd2b None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:20.329647 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-262a1239-1850-41ae-9da1-ea9f60c8fd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:20.397463 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 557/1112] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:51:20.421123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515363fb-b5ec-46cf-bed5-10588549b08c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:20.472757 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:20.525091 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515363fb-b5ec-46cf-bed5-10588549b08c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:20.559824 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f83d8-609e-47f8-b565-709b37ead933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:20.560584 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f83d8-609e-47f8-b565-709b37ead933 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:20.560862 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f83d8-609e-47f8-b565-709b37ead933 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:20.561130 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f83d8-609e-47f8-b565-709b37ead933 None admin] PATH_INFO: `/v3/roles/af11cd5265d14ac7902b1fb3afcc4be7` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:20.562591 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f83d8-609e-47f8-b565-709b37ead933 None admin] RBAC: Authorizing `identity:get_role(role_id=af11cd5265d14ac7902b1fb3afcc4be7)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:20.563991 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f83d8-609e-47f8-b565-709b37ead933 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:20.565349 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 556/1113] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:51:20 2022] GET /identity/v3/roles/af11cd5265d14ac7902b1fb3afcc4be7 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:20.571435 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a1239-1850-41ae-9da1-ea9f60c8fd2b None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:20.602050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:20.673530 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a1239-1850-41ae-9da1-ea9f60c8fd2b None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:20.707090 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c651f2-43a5-4223-89d2-29095cebabb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:20.707955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c651f2-43a5-4223-89d2-29095cebabb9 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:20.708237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c651f2-43a5-4223-89d2-29095cebabb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:20.708514 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c651f2-43a5-4223-89d2-29095cebabb9 None admin] PATH_INFO: `/v3/users/c655df837d704fd3bac9479b1db0703d` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:20.722511 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c651f2-43a5-4223-89d2-29095cebabb9 None admin] RBAC: Authorizing `identity:get_user(user_id=c655df837d704fd3bac9479b1db0703d)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:20.723994 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c651f2-43a5-4223-89d2-29095cebabb9 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:20.725722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 558/1114] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:51:20 2022] GET /identity/v3/users/c655df837d704fd3bac9479b1db0703d => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:20.731564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f83d8-609e-47f8-b565-709b37ead933 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:20.831112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f83d8-609e-47f8-b565-709b37ead933 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:20.850658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:20.868109 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb969af-9697-44df-a9c4-f94d9e0bfc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:20.868753 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb969af-9697-44df-a9c4-f94d9e0bfc89 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:20.869031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb969af-9697-44df-a9c4-f94d9e0bfc89 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:20.869302 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb969af-9697-44df-a9c4-f94d9e0bfc89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:20.950330 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb969af-9697-44df-a9c4-f94d9e0bfc89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:20.951828 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb969af-9697-44df-a9c4-f94d9e0bfc89 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:21.058908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 557/1115] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:51:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 14:51:21.064957 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c651f2-43a5-4223-89d2-29095cebabb9 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:21.155334 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:21.164902 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c651f2-43a5-4223-89d2-29095cebabb9 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:21.199257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af6a51-41d5-445e-ba64-e18f08f76108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:21.200110 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af6a51-41d5-445e-ba64-e18f08f76108 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:21.200391 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af6a51-41d5-445e-ba64-e18f08f76108 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:21.200664 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af6a51-41d5-445e-ba64-e18f08f76108 None admin] PATH_INFO: `/v3/projects/a9ee6e48dec5416cbf54e8f87992aba8` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:21.201840 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af6a51-41d5-445e-ba64-e18f08f76108 None admin] RBAC: Authorizing `identity:get_project(project_id=a9ee6e48dec5416cbf54e8f87992aba8)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:21.203300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af6a51-41d5-445e-ba64-e18f08f76108 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:21.205006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 559/1116] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:51:21 2022] GET /identity/v3/projects/a9ee6e48dec5416cbf54e8f87992aba8 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:21.211439 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb969af-9697-44df-a9c4-f94d9e0bfc89 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:21.229933 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:21.308297 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb969af-9697-44df-a9c4-f94d9e0bfc89 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:21.341913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5a519-bdc8-4751-afd0-a68064a031b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:21.342555 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5a519-bdc8-4751-afd0-a68064a031b7 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:21.342832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5a519-bdc8-4751-afd0-a68064a031b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:21.343130 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5a519-bdc8-4751-afd0-a68064a031b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:21.343952 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5a519-bdc8-4751-afd0-a68064a031b7 None admin] RBAC: Adding query filter params (role.id=af11cd5265d14ac7902b1fb3afcc4be7, scope.project.id=a9ee6e48dec5416cbf54e8f87992aba8, user.id=c655df837d704fd3bac9479b1db0703d) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:21.344256 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5a519-bdc8-4751-afd0-a68064a031b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:21.345671 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5a519-bdc8-4751-afd0-a68064a031b7 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:21.351536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 558/1117] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:51:21 2022] GET /identity/v3/role_assignments?user.id=c655df837d704fd3bac9479b1db0703d&scope.project.id=a9ee6e48dec5416cbf54e8f87992aba8&role.id=af11cd5265d14ac7902b1fb3afcc4be7 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:23.338963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cc8d7-4c47-44cd-95f5-9011ffe6e66d None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:23.339503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cc8d7-4c47-44cd-95f5-9011ffe6e66d None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:23.339503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cc8d7-4c47-44cd-95f5-9011ffe6e66d None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:23.340792 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 560/1118] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:23.346263 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73dc51d-bd10-4014-bd6f-4f777c50666f None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:23.346706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73dc51d-bd10-4014-bd6f-4f777c50666f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:23.346797 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73dc51d-bd10-4014-bd6f-4f777c50666f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:23.364122 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c73dc51d-bd10-4014-bd6f-4f777c50666f None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:23.365318 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c73dc51d-bd10-4014-bd6f-4f777c50666f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:23.432204 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 559/1119] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:51:23.455196 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259cc8d7-4c47-44cd-95f5-9011ffe6e66d None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:23.560220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259cc8d7-4c47-44cd-95f5-9011ffe6e66d None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:23.594793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05df13-fece-49d1-8fc3-6c8090dafab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:23.595592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05df13-fece-49d1-8fc3-6c8090dafab3 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:23.595870 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05df13-fece-49d1-8fc3-6c8090dafab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:23.596142 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05df13-fece-49d1-8fc3-6c8090dafab3 None admin] PATH_INFO: `/v3/roles/af11cd5265d14ac7902b1fb3afcc4be7` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:23.597633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f05df13-fece-49d1-8fc3-6c8090dafab3 None admin] RBAC: Authorizing `identity:get_role(role_id=af11cd5265d14ac7902b1fb3afcc4be7)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:23.599048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f05df13-fece-49d1-8fc3-6c8090dafab3 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:23.600977 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 561/1120] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:51:23 2022] GET /identity/v3/roles/af11cd5265d14ac7902b1fb3afcc4be7 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:23.607027 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73dc51d-bd10-4014-bd6f-4f777c50666f None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:23.621605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:23.706719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73dc51d-bd10-4014-bd6f-4f777c50666f None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:23.745018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2bf4ce-3a75-45ab-a202-c9eeb695cac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:23.745901 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bf4ce-3a75-45ab-a202-c9eeb695cac5 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:23.746178 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bf4ce-3a75-45ab-a202-c9eeb695cac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:23.746451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bf4ce-3a75-45ab-a202-c9eeb695cac5 None admin] PATH_INFO: `/v3/users/c655df837d704fd3bac9479b1db0703d` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:23.747696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2bf4ce-3a75-45ab-a202-c9eeb695cac5 None admin] RBAC: Authorizing `identity:get_user(user_id=c655df837d704fd3bac9479b1db0703d)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:23.749155 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2bf4ce-3a75-45ab-a202-c9eeb695cac5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:23.750846 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 560/1121] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:51:23 2022] GET /identity/v3/users/c655df837d704fd3bac9479b1db0703d => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:23.757746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05df13-fece-49d1-8fc3-6c8090dafab3 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:23.786809 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:23.862488 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05df13-fece-49d1-8fc3-6c8090dafab3 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:23.897323 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80d407-130c-42e0-b7d1-48a5a1ecdb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:23.898183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80d407-130c-42e0-b7d1-48a5a1ecdb2d None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:23.898461 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80d407-130c-42e0-b7d1-48a5a1ecdb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:23.898731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80d407-130c-42e0-b7d1-48a5a1ecdb2d None admin] PATH_INFO: `/v3/projects/a9ee6e48dec5416cbf54e8f87992aba8` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:23.899970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80d407-130c-42e0-b7d1-48a5a1ecdb2d None admin] RBAC: Authorizing `identity:get_project(project_id=a9ee6e48dec5416cbf54e8f87992aba8)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:23.901399 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80d407-130c-42e0-b7d1-48a5a1ecdb2d None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:23.903270 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 562/1122] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:51:23 2022] GET /identity/v3/projects/a9ee6e48dec5416cbf54e8f87992aba8 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:23.909379 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2bf4ce-3a75-45ab-a202-c9eeb695cac5 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:23.988513 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:24.007434 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2bf4ce-3a75-45ab-a202-c9eeb695cac5 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:24.041267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b72949-00e6-47d8-aec5-2fa4dfc700b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:24.042156 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b72949-00e6-47d8-aec5-2fa4dfc700b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:24.042438 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b72949-00e6-47d8-aec5-2fa4dfc700b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:24.042711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b72949-00e6-47d8-aec5-2fa4dfc700b0 None admin] PATH_INFO: `/v3/projects/a9ee6e48dec5416cbf54e8f87992aba8/users/c655df837d704fd3bac9479b1db0703d/roles/af11cd5265d14ac7902b1fb3afcc4be7` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:24.044953 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b72949-00e6-47d8-aec5-2fa4dfc700b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9ee6e48dec5416cbf54e8f87992aba8, user_id=c655df837d704fd3bac9479b1db0703d, role_id=af11cd5265d14ac7902b1fb3afcc4be7)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:24.046391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b72949-00e6-47d8-aec5-2fa4dfc700b0 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:24.141186 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 561/1123] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:51:23 2022] PUT /identity/v3/projects/a9ee6e48dec5416cbf54e8f87992aba8/users/c655df837d704fd3bac9479b1db0703d/roles/af11cd5265d14ac7902b1fb3afcc4be7 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:51:26.135579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eeaf76-6af6-4150-a59b-fc8a187c3d2b None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:26.136108 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eeaf76-6af6-4150-a59b-fc8a187c3d2b None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:26.136108 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eeaf76-6af6-4150-a59b-fc8a187c3d2b None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:26.137493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 563/1124] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:26.143018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e5151-e04e-46cf-b069-505b117fa117 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:26.143456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e5151-e04e-46cf-b069-505b117fa117 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:26.143564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e5151-e04e-46cf-b069-505b117fa117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:26.161988 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b06e5151-e04e-46cf-b069-505b117fa117 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:26.163376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b06e5151-e04e-46cf-b069-505b117fa117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:26.230429 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 562/1125] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:51:26.253496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eeaf76-6af6-4150-a59b-fc8a187c3d2b None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:26.348132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:26.355742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eeaf76-6af6-4150-a59b-fc8a187c3d2b None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:26.391486 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719bb6e-0a3a-4c0f-9b6c-00ec68c97680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:26.392328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719bb6e-0a3a-4c0f-9b6c-00ec68c97680 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:26.392603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719bb6e-0a3a-4c0f-9b6c-00ec68c97680 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:26.392869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719bb6e-0a3a-4c0f-9b6c-00ec68c97680 None admin] PATH_INFO: `/v3/roles/af11cd5265d14ac7902b1fb3afcc4be7` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:26.394344 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719bb6e-0a3a-4c0f-9b6c-00ec68c97680 None admin] RBAC: Authorizing `identity:get_role(role_id=af11cd5265d14ac7902b1fb3afcc4be7)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:26.395734 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719bb6e-0a3a-4c0f-9b6c-00ec68c97680 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:26.397526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 564/1126] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:51:26 2022] GET /identity/v3/roles/af11cd5265d14ac7902b1fb3afcc4be7 => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:26.403710 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e5151-e04e-46cf-b069-505b117fa117 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:26.474148 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:26.504641 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e5151-e04e-46cf-b069-505b117fa117 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:26.539129 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8453da-a795-4aee-b370-a7c1adacbf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:26.540000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8453da-a795-4aee-b370-a7c1adacbf8f None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:26.540278 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8453da-a795-4aee-b370-a7c1adacbf8f None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:26.540550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8453da-a795-4aee-b370-a7c1adacbf8f None admin] PATH_INFO: `/v3/users/c655df837d704fd3bac9479b1db0703d` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:26.541736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8453da-a795-4aee-b370-a7c1adacbf8f None admin] RBAC: Authorizing `identity:get_user(user_id=c655df837d704fd3bac9479b1db0703d)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:26.543188 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8453da-a795-4aee-b370-a7c1adacbf8f None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:26.544813 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 563/1127] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:51:26 2022] GET /identity/v3/users/c655df837d704fd3bac9479b1db0703d => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:26.550615 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719bb6e-0a3a-4c0f-9b6c-00ec68c97680 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:26.648292 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719bb6e-0a3a-4c0f-9b6c-00ec68c97680 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:26.685668 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a3205-5ca7-4e26-ae01-69bc730646c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:26.686321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a3205-5ca7-4e26-ae01-69bc730646c9 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:26.686597 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a3205-5ca7-4e26-ae01-69bc730646c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:26.686864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a3205-5ca7-4e26-ae01-69bc730646c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:26.716590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:26.755095 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9a3205-5ca7-4e26-ae01-69bc730646c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:26.756543 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9a3205-5ca7-4e26-ae01-69bc730646c9 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:26.861093 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 565/1128] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:51:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 14:51:26.867773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8453da-a795-4aee-b370-a7c1adacbf8f None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:26.976582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8453da-a795-4aee-b370-a7c1adacbf8f None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:27.010082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b39ff5-a571-47b8-b297-4e8f814f020e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:27.010876 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b39ff5-a571-47b8-b297-4e8f814f020e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:27.011191 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b39ff5-a571-47b8-b297-4e8f814f020e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:27.011469 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b39ff5-a571-47b8-b297-4e8f814f020e None admin] PATH_INFO: `/v3/projects/a9ee6e48dec5416cbf54e8f87992aba8` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:27.012651 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b39ff5-a571-47b8-b297-4e8f814f020e None admin] RBAC: Authorizing `identity:get_project(project_id=a9ee6e48dec5416cbf54e8f87992aba8)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:27.014095 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b39ff5-a571-47b8-b297-4e8f814f020e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:27.015838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 564/1129] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:51:26 2022] GET /identity/v3/projects/a9ee6e48dec5416cbf54e8f87992aba8 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:27.022512 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a3205-5ca7-4e26-ae01-69bc730646c9 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:27.082380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:27.123119 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a3205-5ca7-4e26-ae01-69bc730646c9 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:27.158186 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e2795-8f4d-46d2-b353-92382923ae0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:27.158853 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e2795-8f4d-46d2-b353-92382923ae0a None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:27.159134 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e2795-8f4d-46d2-b353-92382923ae0a None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:27.159782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e2795-8f4d-46d2-b353-92382923ae0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:27.161063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e2795-8f4d-46d2-b353-92382923ae0a None admin] RBAC: Adding query filter params (role.id=af11cd5265d14ac7902b1fb3afcc4be7, scope.project.id=a9ee6e48dec5416cbf54e8f87992aba8, user.id=c655df837d704fd3bac9479b1db0703d) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:27.161379 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e2795-8f4d-46d2-b353-92382923ae0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:27.164199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e2795-8f4d-46d2-b353-92382923ae0a None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:27.170945 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 566/1130] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:51:27 2022] GET /identity/v3/role_assignments?user.id=c655df837d704fd3bac9479b1db0703d&scope.project.id=a9ee6e48dec5416cbf54e8f87992aba8&role.id=af11cd5265d14ac7902b1fb3afcc4be7 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:29.227684 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f86280-e791-4439-9081-7680840a4a5c None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:29.228182 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f86280-e791-4439-9081-7680840a4a5c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:29.228182 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f86280-e791-4439-9081-7680840a4a5c None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:29.229510 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 565/1131] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:29.234765 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e24ad-1a7e-4eec-9914-d44ec64f064b None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:29.235200 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e24ad-1a7e-4eec-9914-d44ec64f064b None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:29.235366 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e24ad-1a7e-4eec-9914-d44ec64f064b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:29.251995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-776e24ad-1a7e-4eec-9914-d44ec64f064b None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:29.253177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-776e24ad-1a7e-4eec-9914-d44ec64f064b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:29.321125 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 567/1132] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:51:29.345077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f86280-e791-4439-9081-7680840a4a5c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:29.354694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:29.454417 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f86280-e791-4439-9081-7680840a4a5c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:29.489702 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33452e95-2400-483d-8d4f-b0c6c671ac6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:29.491229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33452e95-2400-483d-8d4f-b0c6c671ac6b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:29.491229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33452e95-2400-483d-8d4f-b0c6c671ac6b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:29.491504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33452e95-2400-483d-8d4f-b0c6c671ac6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:29.492697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33452e95-2400-483d-8d4f-b0c6c671ac6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:29.494139 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33452e95-2400-483d-8d4f-b0c6c671ac6b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:29.495802 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 566/1133] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:51:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:29.501998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e24ad-1a7e-4eec-9914-d44ec64f064b None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:29.557407 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:29.599802 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e24ad-1a7e-4eec-9914-d44ec64f064b None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:29.633211 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46fcff-d694-43ce-859a-ce009c438ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:29.633866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46fcff-d694-43ce-859a-ce009c438ff2 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:29.634146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46fcff-d694-43ce-859a-ce009c438ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:29.634418 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46fcff-d694-43ce-859a-ce009c438ff2 None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:29.635169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46fcff-d694-43ce-859a-ce009c438ff2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:29.636600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46fcff-d694-43ce-859a-ce009c438ff2 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:29.722411 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 568/1134] 10.222.0.27 () {60 vars in 1141 bytes} [Sat Jul 16 14:51:29 2022] POST /identity/v3/projects => generated 311 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 14:51:31.786654 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7590c29-c857-429a-a776-a367ad99b00a None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:31.787323 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7590c29-c857-429a-a776-a367ad99b00a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:31.787323 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7590c29-c857-429a-a776-a367ad99b00a None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:31.788594 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 567/1135] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:31.794540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409a582-b15a-43b9-b71b-2076685fef15 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:31.795003 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409a582-b15a-43b9-b71b-2076685fef15 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:31.795176 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6409a582-b15a-43b9-b71b-2076685fef15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:31.812126 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-6409a582-b15a-43b9-b71b-2076685fef15 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:31.813316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-6409a582-b15a-43b9-b71b-2076685fef15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:31.879541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 569/1136] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:51:31.905411 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7590c29-c857-429a-a776-a367ad99b00a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:31.989613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:32.017219 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7590c29-c857-429a-a776-a367ad99b00a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:32.056379 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aaf2fe-819e-4154-9494-c0fbe6fdb29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:32.057082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aaf2fe-819e-4154-9494-c0fbe6fdb29a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:32.057366 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aaf2fe-819e-4154-9494-c0fbe6fdb29a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:32.057641 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aaf2fe-819e-4154-9494-c0fbe6fdb29a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:32.058845 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aaf2fe-819e-4154-9494-c0fbe6fdb29a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:32.060317 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aaf2fe-819e-4154-9494-c0fbe6fdb29a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:32.061956 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 568/1137] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:51:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:32.067873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6409a582-b15a-43b9-b71b-2076685fef15 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:32.109850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:32.173502 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6409a582-b15a-43b9-b71b-2076685fef15 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:32.207376 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9f6d0-510b-43de-8724-223017662690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:32.208038 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9f6d0-510b-43de-8724-223017662690 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:32.208317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9f6d0-510b-43de-8724-223017662690 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:32.208590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9f6d0-510b-43de-8724-223017662690 None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:32.210049 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9f6d0-510b-43de-8724-223017662690 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:32.211578 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9f6d0-510b-43de-8724-223017662690 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:32.289143 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 570/1138] 10.222.0.27 () {60 vars in 1132 bytes} [Sat Jul 16 14:51:32 2022] POST /identity/v3/users => generated 286 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 14:51:34.352001 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1d9e6-f60c-4eda-a595-7a45d1e669ba None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:34.352513 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1d9e6-f60c-4eda-a595-7a45d1e669ba None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:34.352601 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1d9e6-f60c-4eda-a595-7a45d1e669ba None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:34.353915 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 569/1139] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:34.359565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecfe52-8f9e-4cd9-9f28-af4db0cc0e84 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:34.360014 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecfe52-8f9e-4cd9-9f28-af4db0cc0e84 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:34.360114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecfe52-8f9e-4cd9-9f28-af4db0cc0e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:34.377673 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-75ecfe52-8f9e-4cd9-9f28-af4db0cc0e84 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:34.378849 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-75ecfe52-8f9e-4cd9-9f28-af4db0cc0e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:34.448007 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 571/1140] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:51:34.473531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1d9e6-f60c-4eda-a595-7a45d1e669ba None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:34.577969 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1d9e6-f60c-4eda-a595-7a45d1e669ba None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:34.612979 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98836713-2d3a-4ab6-850c-c64c4b4075fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:34.613691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98836713-2d3a-4ab6-850c-c64c4b4075fe None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:34.613977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98836713-2d3a-4ab6-850c-c64c4b4075fe None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:34.614255 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98836713-2d3a-4ab6-850c-c64c4b4075fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:34.624749 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98836713-2d3a-4ab6-850c-c64c4b4075fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:51:34.632491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-98836713-2d3a-4ab6-850c-c64c4b4075fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:51:34.632491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:34.632491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:34.632491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:34.632491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:34.632491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:34.632491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:34.632491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:34.632491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:34.632491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:34.632491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:34.632491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:34.632491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:51:34.632491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:51:34.632491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:51:34.632491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:51:34.633653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:51:34.633653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:51:34.633653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:51:34.633653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:51:34.633653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:34.633653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:34.633653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:34.633653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:34.633653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:34.633653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:34.633653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:34.633653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:34.633653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:34.633653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:34.633653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:34.633653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:34.634694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:34.634694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:34.634694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:34.634694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:34.634694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:51:34.634694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:51:34.634694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:51:34.634694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:51:34.634694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:51:34.634694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:51:34.634694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:51:34.634694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:51:34.634694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 570/1141] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:51:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:51:34.639939 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecfe52-8f9e-4cd9-9f28-af4db0cc0e84 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:34.665931 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:34.742704 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecfe52-8f9e-4cd9-9f28-af4db0cc0e84 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:34.777330 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a356bafa-5165-4f56-9d44-1076722c5d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:34.777994 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356bafa-5165-4f56-9d44-1076722c5d00 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:34.778273 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356bafa-5165-4f56-9d44-1076722c5d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:34.778542 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356bafa-5165-4f56-9d44-1076722c5d00 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:34.779312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a356bafa-5165-4f56-9d44-1076722c5d00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:34.779613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a356bafa-5165-4f56-9d44-1076722c5d00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:34.780976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a356bafa-5165-4f56-9d44-1076722c5d00 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:34.790474 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 572/1142] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:51:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:34.796742 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98836713-2d3a-4ab6-850c-c64c4b4075fe None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:34.799642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:34.901135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98836713-2d3a-4ab6-850c-c64c4b4075fe None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:34.935194 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfae321-6335-40d7-850d-28200458c52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:34.936053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfae321-6335-40d7-850d-28200458c52b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:34.936334 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfae321-6335-40d7-850d-28200458c52b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:34.936610 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfae321-6335-40d7-850d-28200458c52b None admin] PATH_INFO: `/v3/users/f5b24096922d48b59a5eb7fb1ac6fbb2` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:34.950952 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfae321-6335-40d7-850d-28200458c52b None admin] RBAC: Authorizing `identity:get_user(user_id=f5b24096922d48b59a5eb7fb1ac6fbb2)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:34.952392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfae321-6335-40d7-850d-28200458c52b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:34.954356 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 571/1143] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:51:34 2022] GET /identity/v3/users/f5b24096922d48b59a5eb7fb1ac6fbb2 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:34.961479 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a356bafa-5165-4f56-9d44-1076722c5d00 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:35.050383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:35.065585 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a356bafa-5165-4f56-9d44-1076722c5d00 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:35.102491 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3374d322-8fab-4961-9d55-f7354c4f0164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:35.103160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374d322-8fab-4961-9d55-f7354c4f0164 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:35.103445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374d322-8fab-4961-9d55-f7354c4f0164 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:35.103719 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3374d322-8fab-4961-9d55-f7354c4f0164 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:35.175541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3374d322-8fab-4961-9d55-f7354c4f0164 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:35.176934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3374d322-8fab-4961-9d55-f7354c4f0164 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:35.273901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:35.290031 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 573/1144] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:51:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 14:51:35.296172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfae321-6335-40d7-850d-28200458c52b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:35.357644 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:35.399436 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfae321-6335-40d7-850d-28200458c52b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:35.437264 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f42474-ac86-4c40-9050-5650c3b15363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:35.438024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f42474-ac86-4c40-9050-5650c3b15363 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:35.438307 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f42474-ac86-4c40-9050-5650c3b15363 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:35.438578 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f42474-ac86-4c40-9050-5650c3b15363 None admin] PATH_INFO: `/v3/projects/f4e2e86346f94c7aa3c053d7f434e411` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:35.439922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f42474-ac86-4c40-9050-5650c3b15363 None admin] RBAC: Authorizing `identity:get_project(project_id=f4e2e86346f94c7aa3c053d7f434e411)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:35.441569 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f42474-ac86-4c40-9050-5650c3b15363 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:35.444390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 572/1145] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:51:35 2022] GET /identity/v3/projects/f4e2e86346f94c7aa3c053d7f434e411 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:35.450962 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3374d322-8fab-4961-9d55-f7354c4f0164 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:35.550865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3374d322-8fab-4961-9d55-f7354c4f0164 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:35.584920 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd25646-531e-40a3-9f2c-1cf0bc2a5a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:35.585561 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd25646-531e-40a3-9f2c-1cf0bc2a5a09 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:35.585838 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd25646-531e-40a3-9f2c-1cf0bc2a5a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:35.586108 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd25646-531e-40a3-9f2c-1cf0bc2a5a09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:35.587208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd25646-531e-40a3-9f2c-1cf0bc2a5a09 None admin] RBAC: Adding query filter params (role.id=0efeff46573247f3b9f1e2dec2968672, scope.project.id=f4e2e86346f94c7aa3c053d7f434e411, user.id=f5b24096922d48b59a5eb7fb1ac6fbb2) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:35.587513 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd25646-531e-40a3-9f2c-1cf0bc2a5a09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:35.588923 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd25646-531e-40a3-9f2c-1cf0bc2a5a09 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:35.594259 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 574/1146] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:51:35 2022] GET /identity/v3/role_assignments?user.id=f5b24096922d48b59a5eb7fb1ac6fbb2&scope.project.id=f4e2e86346f94c7aa3c053d7f434e411&role.id=0efeff46573247f3b9f1e2dec2968672 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:37.594947 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b092c4-6fac-4d86-9a8b-e9848d91df69 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:37.595498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b092c4-6fac-4d86-9a8b-e9848d91df69 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:37.595498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b092c4-6fac-4d86-9a8b-e9848d91df69 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:37.596942 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 573/1147] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:37.602154 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedb1de-6024-45e1-8a67-017b2a75f66b None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:37.602968 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedb1de-6024-45e1-8a67-017b2a75f66b None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:37.602968 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bedb1de-6024-45e1-8a67-017b2a75f66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:37.620130 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-8bedb1de-6024-45e1-8a67-017b2a75f66b None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:37.621338 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-8bedb1de-6024-45e1-8a67-017b2a75f66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:37.666130 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:37.686994 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 575/1148] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:51:37.709910 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b092c4-6fac-4d86-9a8b-e9848d91df69 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:37.811939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b092c4-6fac-4d86-9a8b-e9848d91df69 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:37.839623 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:37.847483 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0220f4-c8db-43cb-8291-f9c46aa8151b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:37.848203 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0220f4-c8db-43cb-8291-f9c46aa8151b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:37.848482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0220f4-c8db-43cb-8291-f9c46aa8151b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:37.848755 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0220f4-c8db-43cb-8291-f9c46aa8151b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:37.859422 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0220f4-c8db-43cb-8291-f9c46aa8151b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:51:37.867251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-2a0220f4-c8db-43cb-8291-f9c46aa8151b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:51:37.867251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:37.867251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:37.867251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:37.867251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:37.867251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:37.867251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:37.867251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:37.867251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:37.867251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:37.867251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:37.867251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:37.867251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:51:37.867251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:51:37.867251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:51:37.867251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:51:37.868463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:51:37.868463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:51:37.868463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:51:37.868463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:51:37.868463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:37.868463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:37.868463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:37.868463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:37.868463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:37.868463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:37.868463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:37.868463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:37.868463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:37.868463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:37.868463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:37.868463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:37.869558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:37.869558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:37.869558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:37.869558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:37.869558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:51:37.869558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:51:37.869558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:51:37.869558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:51:37.869558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:51:37.869558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:51:37.869558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:51:37.869558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:51:37.869558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 574/1149] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:51:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:51:37.874528 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bedb1de-6024-45e1-8a67-017b2a75f66b None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:37.975425 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bedb1de-6024-45e1-8a67-017b2a75f66b None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:38.010316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed3981-0008-42c9-938c-a5eed1a5adfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:38.011005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed3981-0008-42c9-938c-a5eed1a5adfe None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:38.011288 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed3981-0008-42c9-938c-a5eed1a5adfe None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:38.011562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed3981-0008-42c9-938c-a5eed1a5adfe None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:38.012310 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed3981-0008-42c9-938c-a5eed1a5adfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:38.012615 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed3981-0008-42c9-938c-a5eed1a5adfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:38.013977 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed3981-0008-42c9-938c-a5eed1a5adfe None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:38.022860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 576/1150] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:51:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:38.029035 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0220f4-c8db-43cb-8291-f9c46aa8151b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:38.128050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0220f4-c8db-43cb-8291-f9c46aa8151b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:38.161275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4683b-4b8d-4cb6-93e4-53b08fc2a70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:38.162217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4683b-4b8d-4cb6-93e4-53b08fc2a70a None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:38.162499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4683b-4b8d-4cb6-93e4-53b08fc2a70a None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:38.162776 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4683b-4b8d-4cb6-93e4-53b08fc2a70a None admin] PATH_INFO: `/v3/users/f5b24096922d48b59a5eb7fb1ac6fbb2` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:38.164117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4683b-4b8d-4cb6-93e4-53b08fc2a70a None admin] RBAC: Authorizing `identity:get_user(user_id=f5b24096922d48b59a5eb7fb1ac6fbb2)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:38.165600 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4683b-4b8d-4cb6-93e4-53b08fc2a70a None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:38.167376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 575/1151] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:51:38 2022] GET /identity/v3/users/f5b24096922d48b59a5eb7fb1ac6fbb2 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:38.173572 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed3981-0008-42c9-938c-a5eed1a5adfe None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:38.221479 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:38.281581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed3981-0008-42c9-938c-a5eed1a5adfe None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:38.316408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa84ef1-947b-4843-b11a-82663ad85ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:38.317269 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa84ef1-947b-4843-b11a-82663ad85ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:38.317550 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa84ef1-947b-4843-b11a-82663ad85ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:38.317826 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa84ef1-947b-4843-b11a-82663ad85ce8 None admin] PATH_INFO: `/v3/projects/f4e2e86346f94c7aa3c053d7f434e411` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:38.320045 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa84ef1-947b-4843-b11a-82663ad85ce8 None admin] RBAC: Authorizing `identity:get_project(project_id=f4e2e86346f94c7aa3c053d7f434e411)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:38.321468 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa84ef1-947b-4843-b11a-82663ad85ce8 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:38.323209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 577/1152] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:51:38 2022] GET /identity/v3/projects/f4e2e86346f94c7aa3c053d7f434e411 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:38.329396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4683b-4b8d-4cb6-93e4-53b08fc2a70a None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:38.386006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:38.429413 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4683b-4b8d-4cb6-93e4-53b08fc2a70a None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:38.463283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990241c-55ff-448b-a542-e005a6ae0b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:38.464162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990241c-55ff-448b-a542-e005a6ae0b64 None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:38.464441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990241c-55ff-448b-a542-e005a6ae0b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:38.464713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990241c-55ff-448b-a542-e005a6ae0b64 None admin] PATH_INFO: `/v3/projects/f4e2e86346f94c7aa3c053d7f434e411/users/f5b24096922d48b59a5eb7fb1ac6fbb2/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:38.466652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9990241c-55ff-448b-a542-e005a6ae0b64 None admin] RBAC: Authorizing `identity:create_grant(project_id=f4e2e86346f94c7aa3c053d7f434e411, user_id=f5b24096922d48b59a5eb7fb1ac6fbb2, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:38.468108 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9990241c-55ff-448b-a542-e005a6ae0b64 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:38.552083 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 576/1153] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:51:38 2022] PUT /identity/v3/projects/f4e2e86346f94c7aa3c053d7f434e411/users/f5b24096922d48b59a5eb7fb1ac6fbb2/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:51:40.536031 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8a86e-4a1c-4ffe-ad5e-015aea975308 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:40.536581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8a86e-4a1c-4ffe-ad5e-015aea975308 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:40.536581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8a86e-4a1c-4ffe-ad5e-015aea975308 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:40.538105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 578/1154] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:40.543481 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4539a1f-ee77-4378-9468-d54c384f3032 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:40.543949 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4539a1f-ee77-4378-9468-d54c384f3032 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:40.544061 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4539a1f-ee77-4378-9468-d54c384f3032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:40.561447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-e4539a1f-ee77-4378-9468-d54c384f3032 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:40.562625 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e4539a1f-ee77-4378-9468-d54c384f3032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:40.634978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 577/1155] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:51:40.658166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8a86e-4a1c-4ffe-ad5e-015aea975308 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:40.760877 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8a86e-4a1c-4ffe-ad5e-015aea975308 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:40.772843 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:40.796436 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4b8e5-3e04-4cbf-9f4a-9af1a8dabd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:40.797149 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4b8e5-3e04-4cbf-9f4a-9af1a8dabd34 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:40.797429 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4b8e5-3e04-4cbf-9f4a-9af1a8dabd34 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:40.797700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4b8e5-3e04-4cbf-9f4a-9af1a8dabd34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:40.808791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4b8e5-3e04-4cbf-9f4a-9af1a8dabd34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:51:40.816393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-ffe4b8e5-3e04-4cbf-9f4a-9af1a8dabd34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:51:40.816393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:40.816393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:40.816393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:40.816393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:40.816393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:40.816393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:40.816393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:40.816393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:40.816393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:40.816393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:40.816393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:40.816393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:51:40.816393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:51:40.816393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:51:40.816393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:51:40.817755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:51:40.817755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:51:40.817755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:51:40.817755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:51:40.817755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:40.817755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:40.817755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:40.817755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:40.817755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:40.817755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:40.817755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:40.817755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:40.817755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:40.817755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:40.817755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:40.817755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:40.821795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:40.821795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:40.821795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:40.821795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:40.821795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:51:40.821795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:51:40.821795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:51:40.821795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:51:40.821795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:51:40.821795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:51:40.821795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:51:40.821795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:51:40.821795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 579/1156] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:51:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:51:40.824704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4539a1f-ee77-4378-9468-d54c384f3032 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:40.872510 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:40.930196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4539a1f-ee77-4378-9468-d54c384f3032 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:40.966099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d2658-436e-4ce0-ac6c-f6ad37d3997e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:40.966756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d2658-436e-4ce0-ac6c-f6ad37d3997e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:40.967067 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d2658-436e-4ce0-ac6c-f6ad37d3997e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:40.967338 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d2658-436e-4ce0-ac6c-f6ad37d3997e None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:40.968077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d2658-436e-4ce0-ac6c-f6ad37d3997e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:40.968376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d2658-436e-4ce0-ac6c-f6ad37d3997e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:40.969736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d2658-436e-4ce0-ac6c-f6ad37d3997e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:40.977907 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 578/1157] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:51:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:40.983784 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4b8e5-3e04-4cbf-9f4a-9af1a8dabd34 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:41.085408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4b8e5-3e04-4cbf-9f4a-9af1a8dabd34 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:41.120191 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d90fe-eb82-43e8-b511-c9c5a2e15d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:41.121068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d90fe-eb82-43e8-b511-c9c5a2e15d2e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:41.121348 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d90fe-eb82-43e8-b511-c9c5a2e15d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:41.121620 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d90fe-eb82-43e8-b511-c9c5a2e15d2e None admin] PATH_INFO: `/v3/users/f5b24096922d48b59a5eb7fb1ac6fbb2` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:41.122816 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d90fe-eb82-43e8-b511-c9c5a2e15d2e None admin] RBAC: Authorizing `identity:get_user(user_id=f5b24096922d48b59a5eb7fb1ac6fbb2)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:41.124263 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d90fe-eb82-43e8-b511-c9c5a2e15d2e None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:41.125897 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 580/1158] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:51:40 2022] GET /identity/v3/users/f5b24096922d48b59a5eb7fb1ac6fbb2 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:41.132314 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d2658-436e-4ce0-ac6c-f6ad37d3997e None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:41.234002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d2658-436e-4ce0-ac6c-f6ad37d3997e None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:41.243812 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:41.269009 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc92d3d-dd50-4733-81bc-18e6ee321943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:41.269653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc92d3d-dd50-4733-81bc-18e6ee321943 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:41.269933 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc92d3d-dd50-4733-81bc-18e6ee321943 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:41.270206 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc92d3d-dd50-4733-81bc-18e6ee321943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:41.340091 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc92d3d-dd50-4733-81bc-18e6ee321943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:41.341454 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc92d3d-dd50-4733-81bc-18e6ee321943 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:41.444970 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 579/1159] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:51:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 14:51:41.451222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d90fe-eb82-43e8-b511-c9c5a2e15d2e None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:41.493318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:41.556064 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d90fe-eb82-43e8-b511-c9c5a2e15d2e None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:41.593153 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61cd10-bb6f-4844-a983-4f1ff5b116cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:41.593934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61cd10-bb6f-4844-a983-4f1ff5b116cf None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:41.594213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61cd10-bb6f-4844-a983-4f1ff5b116cf None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:41.594485 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61cd10-bb6f-4844-a983-4f1ff5b116cf None admin] PATH_INFO: `/v3/projects/f4e2e86346f94c7aa3c053d7f434e411` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:41.595689 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61cd10-bb6f-4844-a983-4f1ff5b116cf None admin] RBAC: Authorizing `identity:get_project(project_id=f4e2e86346f94c7aa3c053d7f434e411)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:41.597142 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61cd10-bb6f-4844-a983-4f1ff5b116cf None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:41.598850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 581/1160] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:51:41 2022] GET /identity/v3/projects/f4e2e86346f94c7aa3c053d7f434e411 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:41.605155 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc92d3d-dd50-4733-81bc-18e6ee321943 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:41.615995 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:41.704332 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc92d3d-dd50-4733-81bc-18e6ee321943 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:41.737743 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4214639-c830-4db4-b934-4805b1578537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:41.738392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4214639-c830-4db4-b934-4805b1578537 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:41.738671 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4214639-c830-4db4-b934-4805b1578537 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:41.738967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4214639-c830-4db4-b934-4805b1578537 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:41.740416 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4214639-c830-4db4-b934-4805b1578537 None admin] RBAC: Adding query filter params (role.id=0efeff46573247f3b9f1e2dec2968672, scope.project.id=f4e2e86346f94c7aa3c053d7f434e411, user.id=f5b24096922d48b59a5eb7fb1ac6fbb2) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:41.740734 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4214639-c830-4db4-b934-4805b1578537 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:41.742153 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4214639-c830-4db4-b934-4805b1578537 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:41.747930 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 580/1161] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:51:41 2022] GET /identity/v3/role_assignments?user.id=f5b24096922d48b59a5eb7fb1ac6fbb2&scope.project.id=f4e2e86346f94c7aa3c053d7f434e411&role.id=0efeff46573247f3b9f1e2dec2968672 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:43.773319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e9ad0-c3b8-4713-8932-a8e2b64af3df None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:43.773892 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e9ad0-c3b8-4713-8932-a8e2b64af3df None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:43.773892 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e9ad0-c3b8-4713-8932-a8e2b64af3df None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:43.775243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 582/1162] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:43.780647 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7672a-99a5-490f-bdd1-af113d820486 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:43.781040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7672a-99a5-490f-bdd1-af113d820486 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:43.781205 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7672a-99a5-490f-bdd1-af113d820486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:43.798539 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ccb7672a-99a5-490f-bdd1-af113d820486 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:43.799771 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ccb7672a-99a5-490f-bdd1-af113d820486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:43.866309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 581/1163] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:51:43.889034 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e9ad0-c3b8-4713-8932-a8e2b64af3df None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:43.998052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e9ad0-c3b8-4713-8932-a8e2b64af3df None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:44.005992 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:44.033578 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5c4ae-5e06-4df5-b6a4-11b1b72b08ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:44.034146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5c4ae-5e06-4df5-b6a4-11b1b72b08ca None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:44.034424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5c4ae-5e06-4df5-b6a4-11b1b72b08ca None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:44.034691 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5c4ae-5e06-4df5-b6a4-11b1b72b08ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:44.103497 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5c4ae-5e06-4df5-b6a4-11b1b72b08ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:44.104935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5c4ae-5e06-4df5-b6a4-11b1b72b08ca None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:44.202350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 583/1164] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:51:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 14:51:44.208361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7672a-99a5-490f-bdd1-af113d820486 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:44.211493 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:44.318355 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7672a-99a5-490f-bdd1-af113d820486 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:44.355188 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32de6e4-814c-4486-9a9a-bdca97d70250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:44.355984 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32de6e4-814c-4486-9a9a-bdca97d70250 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:44.356263 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32de6e4-814c-4486-9a9a-bdca97d70250 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:44.356531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32de6e4-814c-4486-9a9a-bdca97d70250 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:44.363417 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32de6e4-814c-4486-9a9a-bdca97d70250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:44.365402 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32de6e4-814c-4486-9a9a-bdca97d70250 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:44.379361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-c32de6e4-814c-4486-9a9a-bdca97d70250 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 16 14:51:44.379361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:44.379361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 16 14:51:44.379361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 16 14:51:44.379361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:51:44.379361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:51:44.379361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:51:44.379361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:51:44.379361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 16 14:51:44.379361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:51:44.379361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:51:44.379361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:51:44.379361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:44.379361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:44.379361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:44.379361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:44.379361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:44.381246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:44.381246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:44.381246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:44.381246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:44.381246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:44.381246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:44.381246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:51:44.381246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:51:44.381246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:51:44.381246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:51:44.381246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:44.381246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:44.381246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:44.381246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:44.381246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:44.381246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:44.382523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:44.382523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:44.382523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:44.382523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:44.382523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:44.382523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:44.382523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:44.382523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:44.382523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:44.382523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:44.382523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 16 14:51:44.382523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:51:44.382523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 16 14:51:44.382523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:51:44.382523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 582/1165] 10.222.0.27 () {56 vars in 1117 bytes} [Sat Jul 16 14:51:44 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:51:44.387557 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5c4ae-5e06-4df5-b6a4-11b1b72b08ca None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:44.395760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:44.485982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5c4ae-5e06-4df5-b6a4-11b1b72b08ca None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:44.520710 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d3bb7d-eb69-4fe8-88e5-6e6500962421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:44.521434 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3bb7d-eb69-4fe8-88e5-6e6500962421 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:44.521716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3bb7d-eb69-4fe8-88e5-6e6500962421 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:44.521991 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3bb7d-eb69-4fe8-88e5-6e6500962421 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:44.522710 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d3bb7d-eb69-4fe8-88e5-6e6500962421 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:44.523042 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d3bb7d-eb69-4fe8-88e5-6e6500962421 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:44.524409 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d3bb7d-eb69-4fe8-88e5-6e6500962421 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:44.531566 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 584/1166] 10.222.0.27 () {56 vars in 1115 bytes} [Sat Jul 16 14:51:44 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:46.542024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b8343-30e2-459d-a52d-a145fe0116ee None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:46.542577 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b8343-30e2-459d-a52d-a145fe0116ee None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:46.542577 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b8343-30e2-459d-a52d-a145fe0116ee None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:46.543970 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 583/1167] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:46.549138 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85f915-e2bf-4fd4-8c35-bd0321647a55 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:46.549591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85f915-e2bf-4fd4-8c35-bd0321647a55 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:46.549696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85f915-e2bf-4fd4-8c35-bd0321647a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:46.567612 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-4f85f915-e2bf-4fd4-8c35-bd0321647a55 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:46.569703 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-4f85f915-e2bf-4fd4-8c35-bd0321647a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:46.637833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:46.642588 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 585/1168] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:51:46.666127 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b8343-30e2-459d-a52d-a145fe0116ee None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:46.731012 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:46.771389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b8343-30e2-459d-a52d-a145fe0116ee None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:46.804994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf5288-70bf-4a07-bf67-2b334404b4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:46.805570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf5288-70bf-4a07-bf67-2b334404b4a5 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:46.805851 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf5288-70bf-4a07-bf67-2b334404b4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:46.806126 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf5288-70bf-4a07-bf67-2b334404b4a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:46.806867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf5288-70bf-4a07-bf67-2b334404b4a5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:46.808259 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf5288-70bf-4a07-bf67-2b334404b4a5 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:46.878512 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 584/1169] 10.222.0.27 () {60 vars in 1138 bytes} [Sat Jul 16 14:51:46 2022] POST /identity/v3/domains => generated 267 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:51:48.924641 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbffe86-f215-4207-a5c5-1c8f0f196992 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:48.925145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbffe86-f215-4207-a5c5-1c8f0f196992 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:48.925258 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbffe86-f215-4207-a5c5-1c8f0f196992 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:48.926545 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 586/1170] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 14:51:48.932008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886c000-6531-47e4-9b48-199496e58b10 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:48.932439 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886c000-6531-47e4-9b48-199496e58b10 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:48.932538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886c000-6531-47e4-9b48-199496e58b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:48.949592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-9886c000-6531-47e4-9b48-199496e58b10 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:48.950786 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9886c000-6531-47e4-9b48-199496e58b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:49.018315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 585/1171] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:51:49.043472 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbffe86-f215-4207-a5c5-1c8f0f196992 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:49.153283 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbffe86-f215-4207-a5c5-1c8f0f196992 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:49.189606 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8477f90-6eb6-40ab-ad71-3006a9afca52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:49.190490 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8477f90-6eb6-40ab-ad71-3006a9afca52 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:49.190767 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8477f90-6eb6-40ab-ad71-3006a9afca52 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:49.191067 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8477f90-6eb6-40ab-ad71-3006a9afca52 None admin] PATH_INFO: `/v3/domains/19b6793b5f7746cb9457f3ad166c19e2` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:49.202676 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8477f90-6eb6-40ab-ad71-3006a9afca52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=19b6793b5f7746cb9457f3ad166c19e2)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:49.204178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8477f90-6eb6-40ab-ad71-3006a9afca52 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:49.205947 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 587/1172] 10.222.0.27 () {56 vars in 1183 bytes} [Sat Jul 16 14:51:49 2022] GET /identity/v3/domains/19b6793b5f7746cb9457f3ad166c19e2 => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:49.212384 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886c000-6531-47e4-9b48-199496e58b10 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:49.266243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:49.313995 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886c000-6531-47e4-9b48-199496e58b10 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:49.348640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6b301-127c-403c-af0c-f348e6921ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:49.350019 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6b301-127c-403c-af0c-f348e6921ca7 None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:49.350302 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6b301-127c-403c-af0c-f348e6921ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:49.350574 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6b301-127c-403c-af0c-f348e6921ca7 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:49.351309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f6b301-127c-403c-af0c-f348e6921ca7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:49.352744 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f6b301-127c-403c-af0c-f348e6921ca7 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:49.467009 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 586/1173] 10.222.0.27 () {60 vars in 1141 bytes} [Sat Jul 16 14:51:49 2022] POST /identity/v3/projects => generated 361 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:51:51.538229 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9ba65-cbaf-454f-bfa6-6cc3a12282a4 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:51.538782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9ba65-cbaf-454f-bfa6-6cc3a12282a4 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:51.538782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9ba65-cbaf-454f-bfa6-6cc3a12282a4 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:51.540239 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 588/1174] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:51.545506 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03ac7b-2bff-4e71-94b1-7564f5ca41cf None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:51.545933 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03ac7b-2bff-4e71-94b1-7564f5ca41cf None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:51.546056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03ac7b-2bff-4e71-94b1-7564f5ca41cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:51.563641 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5a03ac7b-2bff-4e71-94b1-7564f5ca41cf None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:51.564826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5a03ac7b-2bff-4e71-94b1-7564f5ca41cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:51.641036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 587/1175] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:51:51.667113 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9ba65-cbaf-454f-bfa6-6cc3a12282a4 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:51.732434 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:51.776350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9ba65-cbaf-454f-bfa6-6cc3a12282a4 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:51.810683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a3bf0-49a2-4e9a-ad61-d01e0b971d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:51.811488 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a3bf0-49a2-4e9a-ad61-d01e0b971d19 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:51.811765 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a3bf0-49a2-4e9a-ad61-d01e0b971d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:51.812034 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a3bf0-49a2-4e9a-ad61-d01e0b971d19 None admin] PATH_INFO: `/v3/domains/19b6793b5f7746cb9457f3ad166c19e2` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:51.813199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a3bf0-49a2-4e9a-ad61-d01e0b971d19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=19b6793b5f7746cb9457f3ad166c19e2)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:51.814616 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a3bf0-49a2-4e9a-ad61-d01e0b971d19 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:51.816259 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 589/1176] 10.222.0.27 () {56 vars in 1183 bytes} [Sat Jul 16 14:51:51 2022] GET /identity/v3/domains/19b6793b5f7746cb9457f3ad166c19e2 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:51.825276 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03ac7b-2bff-4e71-94b1-7564f5ca41cf None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:51.861321 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:51.928937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03ac7b-2bff-4e71-94b1-7564f5ca41cf None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:51.963192 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436bb23f-5a20-4db2-9c46-0e6c2dbaff2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:51.963847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bb23f-5a20-4db2-9c46-0e6c2dbaff2e None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:51.964123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bb23f-5a20-4db2-9c46-0e6c2dbaff2e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:51.964393 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436bb23f-5a20-4db2-9c46-0e6c2dbaff2e None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:51.965092 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436bb23f-5a20-4db2-9c46-0e6c2dbaff2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:51.966519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436bb23f-5a20-4db2-9c46-0e6c2dbaff2e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:52.061371 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 588/1177] 10.222.0.27 () {60 vars in 1132 bytes} [Sat Jul 16 14:51:51 2022] POST /identity/v3/users => generated 311 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:51:54.103895 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfe1d2-78c5-41d0-bc5c-2e078ef63c6f None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:54.104415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfe1d2-78c5-41d0-bc5c-2e078ef63c6f None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:54.104415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfe1d2-78c5-41d0-bc5c-2e078ef63c6f None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:54.105795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 590/1178] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:54.110999 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73f4ee-8f55-4a29-bb22-f98bcaa32290 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:54.111381 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73f4ee-8f55-4a29-bb22-f98bcaa32290 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:54.111553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73f4ee-8f55-4a29-bb22-f98bcaa32290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:54.129605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-8b73f4ee-8f55-4a29-bb22-f98bcaa32290 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:54.131767 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-8b73f4ee-8f55-4a29-bb22-f98bcaa32290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:54.208922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 589/1179] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:51:54.235032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfe1d2-78c5-41d0-bc5c-2e078ef63c6f None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:54.359669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfe1d2-78c5-41d0-bc5c-2e078ef63c6f None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:54.392187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:54.402143 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61f8fc-0268-41a0-9993-7155fcc00516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:54.414552 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61f8fc-0268-41a0-9993-7155fcc00516 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:54.414552 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61f8fc-0268-41a0-9993-7155fcc00516 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:54.414552 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61f8fc-0268-41a0-9993-7155fcc00516 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:54.415439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61f8fc-0268-41a0-9993-7155fcc00516 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:51:54.423293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-7a61f8fc-0268-41a0-9993-7155fcc00516 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:51:54.423293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:54.423293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:54.423293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:54.423293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:54.423293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:54.423293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:54.423293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:54.423293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:54.423293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:54.423293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:54.423293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:54.423293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:51:54.423293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:51:54.423293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:51:54.423293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:51:54.424653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:51:54.424653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:51:54.424653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:51:54.424653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:51:54.424653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:54.424653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:54.424653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:54.424653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:54.424653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:54.424653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:54.424653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:54.424653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:54.424653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:54.424653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:54.424653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:54.424653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:54.425946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:54.425946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:54.425946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:54.425946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:54.425946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:51:54.425946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:51:54.425946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:51:54.425946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:51:54.425946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:51:54.425946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:51:54.425946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:51:54.425946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:51:54.425946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 591/1180] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:51:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:51:54.431520 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73f4ee-8f55-4a29-bb22-f98bcaa32290 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:54.454710 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:54.554051 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73f4ee-8f55-4a29-bb22-f98bcaa32290 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:54.590787 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce17126-3f1e-491a-8503-06de00609f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:54.591498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce17126-3f1e-491a-8503-06de00609f2e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:54.591780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce17126-3f1e-491a-8503-06de00609f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:54.592053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce17126-3f1e-491a-8503-06de00609f2e None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:54.592810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce17126-3f1e-491a-8503-06de00609f2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:54.593118 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce17126-3f1e-491a-8503-06de00609f2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:54.594505 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce17126-3f1e-491a-8503-06de00609f2e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:54.604194 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 590/1181] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:51:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:54.611387 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61f8fc-0268-41a0-9993-7155fcc00516 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:54.725555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61f8fc-0268-41a0-9993-7155fcc00516 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:54.763224 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29144871-4347-40e3-9732-69502b7ef075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:54.764093 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29144871-4347-40e3-9732-69502b7ef075 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:54.764371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29144871-4347-40e3-9732-69502b7ef075 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:54.764644 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29144871-4347-40e3-9732-69502b7ef075 None admin] PATH_INFO: `/v3/users/3ec7faf03cd146d0989ede6d34e3b5c9` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:54.780448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29144871-4347-40e3-9732-69502b7ef075 None admin] RBAC: Authorizing `identity:get_user(user_id=3ec7faf03cd146d0989ede6d34e3b5c9)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:54.781971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29144871-4347-40e3-9732-69502b7ef075 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:54.783765 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 592/1182] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:51:54 2022] GET /identity/v3/users/3ec7faf03cd146d0989ede6d34e3b5c9 => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:54.790315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce17126-3f1e-491a-8503-06de00609f2e None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:54.895388 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:54.918640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce17126-3f1e-491a-8503-06de00609f2e None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:54.956057 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e358c8-748a-4f74-85e5-d814d0e56df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:54.956708 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e358c8-748a-4f74-85e5-d814d0e56df1 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:54.956993 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e358c8-748a-4f74-85e5-d814d0e56df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:54.957277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e358c8-748a-4f74-85e5-d814d0e56df1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:55.034784 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e358c8-748a-4f74-85e5-d814d0e56df1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:55.036302 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e358c8-748a-4f74-85e5-d814d0e56df1 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:55.134713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:55.157879 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 591/1183] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:51:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 14:51:55.165713 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29144871-4347-40e3-9732-69502b7ef075 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:55.213941 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:55.266838 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29144871-4347-40e3-9732-69502b7ef075 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:55.301300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc199841-f097-4d68-8503-110f993e6d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:55.302924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc199841-f097-4d68-8503-110f993e6d53 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:55.303218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc199841-f097-4d68-8503-110f993e6d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:55.303490 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc199841-f097-4d68-8503-110f993e6d53 None admin] PATH_INFO: `/v3/projects/17edb8ba1bd04db597ca9edc6412b5e0` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:55.304690 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc199841-f097-4d68-8503-110f993e6d53 None admin] RBAC: Authorizing `identity:get_project(project_id=17edb8ba1bd04db597ca9edc6412b5e0)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:55.306123 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc199841-f097-4d68-8503-110f993e6d53 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:55.307865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 593/1184] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:51:55 2022] GET /identity/v3/projects/17edb8ba1bd04db597ca9edc6412b5e0 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:55.314081 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e358c8-748a-4f74-85e5-d814d0e56df1 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:55.412603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e358c8-748a-4f74-85e5-d814d0e56df1 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:55.446466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b67b7-9b53-4367-a188-98f0e14d5e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:55.447146 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b67b7-9b53-4367-a188-98f0e14d5e44 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:55.447426 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b67b7-9b53-4367-a188-98f0e14d5e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:55.447696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b67b7-9b53-4367-a188-98f0e14d5e44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:55.450138 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b67b7-9b53-4367-a188-98f0e14d5e44 None admin] RBAC: Adding query filter params (role.id=0efeff46573247f3b9f1e2dec2968672, scope.project.id=17edb8ba1bd04db597ca9edc6412b5e0, user.id=3ec7faf03cd146d0989ede6d34e3b5c9) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:55.450138 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b67b7-9b53-4367-a188-98f0e14d5e44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:55.450368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b67b7-9b53-4367-a188-98f0e14d5e44 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:55.456104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 592/1185] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:51:55 2022] GET /identity/v3/role_assignments?user.id=3ec7faf03cd146d0989ede6d34e3b5c9&scope.project.id=17edb8ba1bd04db597ca9edc6412b5e0&role.id=0efeff46573247f3b9f1e2dec2968672 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:57.442005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2736d17-8687-4407-aa98-4603528effd9 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:57.442567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2736d17-8687-4407-aa98-4603528effd9 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:57.442567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2736d17-8687-4407-aa98-4603528effd9 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:57.444483 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 594/1186] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:51:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:51:57.449931 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca56468-391a-4a81-bc95-2d1255bba2fd None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:57.450371 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca56468-391a-4a81-bc95-2d1255bba2fd None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:57.450483 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca56468-391a-4a81-bc95-2d1255bba2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:57.468926 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-eca56468-391a-4a81-bc95-2d1255bba2fd None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:51:57.470125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-eca56468-391a-4a81-bc95-2d1255bba2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:51:57.512354 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:57.537343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 593/1187] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:51:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:51:57.559622 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2736d17-8687-4407-aa98-4603528effd9 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:57.668313 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2736d17-8687-4407-aa98-4603528effd9 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:57.682275 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:57.703263 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd2524-1621-4a3f-9726-22ec6ca53003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:57.703974 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd2524-1621-4a3f-9726-22ec6ca53003 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:57.704248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd2524-1621-4a3f-9726-22ec6ca53003 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:57.704519 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd2524-1621-4a3f-9726-22ec6ca53003 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:57.716295 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd2524-1621-4a3f-9726-22ec6ca53003 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:51:57.723855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-6dcd2524-1621-4a3f-9726-22ec6ca53003 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:51:57.723855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:51:57.723855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:51:57.723855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:51:57.723855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:51:57.723855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:51:57.723855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:51:57.723855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:51:57.723855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:51:57.723855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:51:57.723855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:51:57.723855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:51:57.723855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:51:57.723855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:51:57.723855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:51:57.723855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:51:57.724997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:51:57.724997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:51:57.724997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:51:57.724997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:51:57.724997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:51:57.724997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:51:57.724997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:51:57.724997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:51:57.724997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:51:57.724997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:51:57.724997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:51:57.724997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:51:57.724997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:51:57.724997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:51:57.724997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:51:57.724997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:51:57.726159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:51:57.726159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:51:57.726159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:51:57.726159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:51:57.726159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:51:57.726159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:51:57.726159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:51:57.726159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:51:57.726159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:51:57.726159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:51:57.726159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:51:57.726159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:51:57.726159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 595/1188] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:51:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:51:57.731346 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca56468-391a-4a81-bc95-2d1255bba2fd None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:57.834430 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca56468-391a-4a81-bc95-2d1255bba2fd None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:57.868924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2819eb-c7b1-4caa-97e3-d4ca343e85f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:57.869585 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2819eb-c7b1-4caa-97e3-d4ca343e85f3 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:57.869863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2819eb-c7b1-4caa-97e3-d4ca343e85f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:57.870134 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2819eb-c7b1-4caa-97e3-d4ca343e85f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:57.870875 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2819eb-c7b1-4caa-97e3-d4ca343e85f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:51:57.871204 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2819eb-c7b1-4caa-97e3-d4ca343e85f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:57.872570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2819eb-c7b1-4caa-97e3-d4ca343e85f3 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:57.882526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 594/1189] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:51:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:57.886755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd2524-1621-4a3f-9726-22ec6ca53003 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:57.988706 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd2524-1621-4a3f-9726-22ec6ca53003 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:58.025448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05533fb2-f522-42f7-b9f1-cb355051ee1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:58.026328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05533fb2-f522-42f7-b9f1-cb355051ee1b None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:58.026608 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05533fb2-f522-42f7-b9f1-cb355051ee1b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:58.026877 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05533fb2-f522-42f7-b9f1-cb355051ee1b None admin] PATH_INFO: `/v3/users/3ec7faf03cd146d0989ede6d34e3b5c9` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:58.028234 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05533fb2-f522-42f7-b9f1-cb355051ee1b None admin] RBAC: Authorizing `identity:get_user(user_id=3ec7faf03cd146d0989ede6d34e3b5c9)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:58.029708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05533fb2-f522-42f7-b9f1-cb355051ee1b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:58.031944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 596/1190] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:51:57 2022] GET /identity/v3/users/3ec7faf03cd146d0989ede6d34e3b5c9 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:58.040317 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2819eb-c7b1-4caa-97e3-d4ca343e85f3 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:58.082331 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:58.146440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2819eb-c7b1-4caa-97e3-d4ca343e85f3 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:58.180875 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a59e1-2dd5-4a87-a8b0-f5fd54e9f0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:58.181723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a59e1-2dd5-4a87-a8b0-f5fd54e9f0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:58.182002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a59e1-2dd5-4a87-a8b0-f5fd54e9f0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:58.182276 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a59e1-2dd5-4a87-a8b0-f5fd54e9f0b2 None admin] PATH_INFO: `/v3/projects/17edb8ba1bd04db597ca9edc6412b5e0` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:58.183475 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a59e1-2dd5-4a87-a8b0-f5fd54e9f0b2 None admin] RBAC: Authorizing `identity:get_project(project_id=17edb8ba1bd04db597ca9edc6412b5e0)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:58.184907 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a59e1-2dd5-4a87-a8b0-f5fd54e9f0b2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:58.186603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 595/1191] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:51:58 2022] GET /identity/v3/projects/17edb8ba1bd04db597ca9edc6412b5e0 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:51:58.192902 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05533fb2-f522-42f7-b9f1-cb355051ee1b None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:51:58.236261 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:51:58.293558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05533fb2-f522-42f7-b9f1-cb355051ee1b None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:51:58.327401 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a60c4-adf6-4ba2-98d0-cb0899c7f656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:51:58.328275 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a60c4-adf6-4ba2-98d0-cb0899c7f656 None admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:51:58.328552 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a60c4-adf6-4ba2-98d0-cb0899c7f656 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:51:58.328818 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a60c4-adf6-4ba2-98d0-cb0899c7f656 None admin] PATH_INFO: `/v3/projects/17edb8ba1bd04db597ca9edc6412b5e0/users/3ec7faf03cd146d0989ede6d34e3b5c9/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:51:58.330741 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a60c4-adf6-4ba2-98d0-cb0899c7f656 None admin] RBAC: Authorizing `identity:create_grant(project_id=17edb8ba1bd04db597ca9edc6412b5e0, user_id=3ec7faf03cd146d0989ede6d34e3b5c9, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:51:58.332189 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a60c4-adf6-4ba2-98d0-cb0899c7f656 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:51:58.403778 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 597/1192] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:51:58 2022] PUT /identity/v3/projects/17edb8ba1bd04db597ca9edc6412b5e0/users/3ec7faf03cd146d0989ede6d34e3b5c9/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:52:00.421780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a263d1-5d85-439a-bc1a-7505faee3541 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:52:00.422343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a263d1-5d85-439a-bc1a-7505faee3541 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:52:00.422343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a263d1-5d85-439a-bc1a-7505faee3541 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:52:00.423727 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 596/1193] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:52:00.430480 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd07276-997f-42f8-ae2a-454c39b73fd1 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:52:00.431025 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd07276-997f-42f8-ae2a-454c39b73fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:52:00.431535 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd07276-997f-42f8-ae2a-454c39b73fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:52:00.459543 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-2fd07276-997f-42f8-ae2a-454c39b73fd1 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:52:00.461182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-2fd07276-997f-42f8-ae2a-454c39b73fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:52:00.541370 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 598/1194] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:52:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:52:00.565257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a263d1-5d85-439a-bc1a-7505faee3541 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:52:00.669912 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a263d1-5d85-439a-bc1a-7505faee3541 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:52:00.677443 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:52:00.705383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7da73-6b2a-4b0b-99bd-a74bc6bfc519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:52:00.706170 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7da73-6b2a-4b0b-99bd-a74bc6bfc519 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:52:00.706452 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7da73-6b2a-4b0b-99bd-a74bc6bfc519 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:52:00.706727 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7da73-6b2a-4b0b-99bd-a74bc6bfc519 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:52:00.718282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7da73-6b2a-4b0b-99bd-a74bc6bfc519 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:52:00.726686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-f8f7da73-6b2a-4b0b-99bd-a74bc6bfc519 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:52:00.726686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:52:00.726686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:52:00.726686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:52:00.726686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:52:00.726686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:52:00.726686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:52:00.726686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:52:00.726686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:52:00.726686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:52:00.726686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:52:00.726686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:52:00.726686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:52:00.726686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:52:00.726686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:52:00.726686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:52:00.727861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:52:00.727861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:52:00.727861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:52:00.727861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:52:00.727861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:52:00.727861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:52:00.727861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:52:00.727861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:52:00.727861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:52:00.727861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:52:00.727861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:52:00.727861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:52:00.727861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:52:00.727861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:52:00.727861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:52:00.727861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:52:00.728728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:52:00.728728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:52:00.728728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:52:00.728728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:52:00.728728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:52:00.728728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:52:00.728728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:52:00.728728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:52:00.728728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:52:00.728728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:52:00.728728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:52:00.728728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:52:00.728728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 597/1195] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:52:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:52:00.734209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd07276-997f-42f8-ae2a-454c39b73fd1 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:52:00.767050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:52:00.838598 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd07276-997f-42f8-ae2a-454c39b73fd1 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:52:00.874696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37f37e-6775-41df-93d6-3be92659d145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:52:00.875378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37f37e-6775-41df-93d6-3be92659d145 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:52:00.875655 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37f37e-6775-41df-93d6-3be92659d145 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:52:00.875925 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37f37e-6775-41df-93d6-3be92659d145 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:52:00.876664 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37f37e-6775-41df-93d6-3be92659d145 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:52:00.876964 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37f37e-6775-41df-93d6-3be92659d145 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:52:00.878338 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37f37e-6775-41df-93d6-3be92659d145 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:52:00.891746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 599/1196] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:52:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:52:00.898113 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7da73-6b2a-4b0b-99bd-a74bc6bfc519 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:52:00.999664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7da73-6b2a-4b0b-99bd-a74bc6bfc519 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:52:01.034189 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcca3c1-c69e-408a-a2db-b589e4515e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:52:01.035081 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcca3c1-c69e-408a-a2db-b589e4515e01 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:52:01.035359 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcca3c1-c69e-408a-a2db-b589e4515e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:52:01.035627 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcca3c1-c69e-408a-a2db-b589e4515e01 None admin] PATH_INFO: `/v3/users/3ec7faf03cd146d0989ede6d34e3b5c9` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:52:01.037407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcca3c1-c69e-408a-a2db-b589e4515e01 None admin] RBAC: Authorizing `identity:get_user(user_id=3ec7faf03cd146d0989ede6d34e3b5c9)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:52:01.038839 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcca3c1-c69e-408a-a2db-b589e4515e01 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:52:01.040507 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 598/1197] 10.222.0.27 () {56 vars in 1177 bytes} [Sat Jul 16 14:52:00 2022] GET /identity/v3/users/3ec7faf03cd146d0989ede6d34e3b5c9 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:52:01.046576 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37f37e-6775-41df-93d6-3be92659d145 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:52:01.140375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:52:01.147888 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37f37e-6775-41df-93d6-3be92659d145 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:52:01.182076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbaa16-a310-4372-8415-eac9039ad78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:52:01.182723 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbaa16-a310-4372-8415-eac9039ad78f None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:52:01.183029 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbaa16-a310-4372-8415-eac9039ad78f None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:52:01.183300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbaa16-a310-4372-8415-eac9039ad78f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:52:01.256142 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdbaa16-a310-4372-8415-eac9039ad78f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:52:01.257904 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdbaa16-a310-4372-8415-eac9039ad78f None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:52:01.357633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:52:01.365510 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 600/1198] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:52:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 14:52:01.371775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcca3c1-c69e-408a-a2db-b589e4515e01 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:52:01.409447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:52:01.478284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcca3c1-c69e-408a-a2db-b589e4515e01 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:52:01.512646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e4acc-babc-4729-b728-2eca41f8a067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:52:01.513427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e4acc-babc-4729-b728-2eca41f8a067 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:52:01.513707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e4acc-babc-4729-b728-2eca41f8a067 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:52:01.513980 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e4acc-babc-4729-b728-2eca41f8a067 None admin] PATH_INFO: `/v3/projects/17edb8ba1bd04db597ca9edc6412b5e0` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:52:01.515212 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e4acc-babc-4729-b728-2eca41f8a067 None admin] RBAC: Authorizing `identity:get_project(project_id=17edb8ba1bd04db597ca9edc6412b5e0)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:52:01.516642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e4acc-babc-4729-b728-2eca41f8a067 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:52:01.518343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 599/1199] 10.222.0.27 () {56 vars in 1186 bytes} [Sat Jul 16 14:52:01 2022] GET /identity/v3/projects/17edb8ba1bd04db597ca9edc6412b5e0 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:52:01.525432 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbaa16-a310-4372-8415-eac9039ad78f None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:52:01.626190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbaa16-a310-4372-8415-eac9039ad78f None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:52:01.661782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd51d43-fee9-4252-982b-6f8a4582f3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:52:01.662526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd51d43-fee9-4252-982b-6f8a4582f3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:52:01.662805 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd51d43-fee9-4252-982b-6f8a4582f3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:52:01.663106 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd51d43-fee9-4252-982b-6f8a4582f3b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:52:01.663944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd51d43-fee9-4252-982b-6f8a4582f3b3 None admin] RBAC: Adding query filter params (role.id=0efeff46573247f3b9f1e2dec2968672, scope.project.id=17edb8ba1bd04db597ca9edc6412b5e0, user.id=3ec7faf03cd146d0989ede6d34e3b5c9) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:52:01.664253 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd51d43-fee9-4252-982b-6f8a4582f3b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:52:01.665688 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd51d43-fee9-4252-982b-6f8a4582f3b3 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:52:01.672029 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 601/1200] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:52:01 2022] GET /identity/v3/role_assignments?user.id=3ec7faf03cd146d0989ede6d34e3b5c9&scope.project.id=17edb8ba1bd04db597ca9edc6412b5e0&role.id=0efeff46573247f3b9f1e2dec2968672 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:52:11.508219 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=49 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:52:11.669036 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=71 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:21.757929 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cf87e-1b28-403a-baec-86a1ae213a3b None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:21.758554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cf87e-1b28-403a-baec-86a1ae213a3b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:21.758554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cf87e-1b28-403a-baec-86a1ae213a3b None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:21.759928 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 600/1201] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:53:21.764959 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81c7b7-0da2-4272-852e-7eafe9303b8a None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:21.765992 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81c7b7-0da2-4272-852e-7eafe9303b8a None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:21.766212 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81c7b7-0da2-4272-852e-7eafe9303b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:21.783210 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-ea81c7b7-0da2-4272-852e-7eafe9303b8a None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:53:21.786434 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ea81c7b7-0da2-4272-852e-7eafe9303b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:53:21.850372 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 602/1202] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:53:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:53:21.874481 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96cf87e-1b28-403a-baec-86a1ae213a3b None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:21.980026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96cf87e-1b28-403a-baec-86a1ae213a3b None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:22.016521 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0a7cf-c46f-4293-975c-e3aecc0d9867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:22.017233 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0a7cf-c46f-4293-975c-e3aecc0d9867 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:22.017509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0a7cf-c46f-4293-975c-e3aecc0d9867 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:22.017775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0a7cf-c46f-4293-975c-e3aecc0d9867 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:22.028067 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e0a7cf-c46f-4293-975c-e3aecc0d9867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:22.029547 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e0a7cf-c46f-4293-975c-e3aecc0d9867 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:22.048232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-65e0a7cf-c46f-4293-975c-e3aecc0d9867 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:22.048232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:22.048232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:22.048232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:22.048232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:22.048232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:22.048232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:22.048232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:22.048232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:22.048232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:22.048232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:22.048232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:22.048232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:53:22.048232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:53:22.048232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:53:22.049310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:53:22.049310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:22.049310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:22.049310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:22.049310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:22.049310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:22.049310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:22.049310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:22.049310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:22.049310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:22.049310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:22.049310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:22.049310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:22.049310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:22.049310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:22.049310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:22.049310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:22.050339 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:53:22.050339 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:53:22.050339 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:22.050339 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:53:22.050339 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 601/1203] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:22.055916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81c7b7-0da2-4272-852e-7eafe9303b8a None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:22.160793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81c7b7-0da2-4272-852e-7eafe9303b8a None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:22.190366 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:22.196491 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a138758-b974-43ab-b6bf-2b49348e88e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:22.197151 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a138758-b974-43ab-b6bf-2b49348e88e0 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:22.197432 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a138758-b974-43ab-b6bf-2b49348e88e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:22.197722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a138758-b974-43ab-b6bf-2b49348e88e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:22.198471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a138758-b974-43ab-b6bf-2b49348e88e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:22.198754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a138758-b974-43ab-b6bf-2b49348e88e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:22.200167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a138758-b974-43ab-b6bf-2b49348e88e0 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:22.212088 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 603/1204] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:22.217938 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0a7cf-c46f-4293-975c-e3aecc0d9867 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:22.272460 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:22.321295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0a7cf-c46f-4293-975c-e3aecc0d9867 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:22.356292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac29018-e72e-4db9-acd2-a564617c540e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:22.356951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac29018-e72e-4db9-acd2-a564617c540e None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:22.357228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac29018-e72e-4db9-acd2-a564617c540e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:22.357499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac29018-e72e-4db9-acd2-a564617c540e None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:22.358209 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac29018-e72e-4db9-acd2-a564617c540e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:22.360375 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac29018-e72e-4db9-acd2-a564617c540e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:22.463940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 602/1205] 10.222.0.27 () {60 vars in 1132 bytes} [Sat Jul 16 14:53:22 2022] POST /identity/v3/users => generated 251 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:53:24.577197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2939a21-acb2-4e2b-b1dc-fb38ca847e3e None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:24.577767 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2939a21-acb2-4e2b-b1dc-fb38ca847e3e None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:24.577876 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2939a21-acb2-4e2b-b1dc-fb38ca847e3e None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:24.579436 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 604/1206] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:53:24.584865 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace1633-b3d9-4ffe-8e97-1f3ca5da29b0 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:24.585303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace1633-b3d9-4ffe-8e97-1f3ca5da29b0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:24.587442 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace1633-b3d9-4ffe-8e97-1f3ca5da29b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:24.603031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-9ace1633-b3d9-4ffe-8e97-1f3ca5da29b0 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:53:24.604229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9ace1633-b3d9-4ffe-8e97-1f3ca5da29b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:53:24.673943 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 603/1207] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:53:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:53:24.697833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2939a21-acb2-4e2b-b1dc-fb38ca847e3e None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:24.809598 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2939a21-acb2-4e2b-b1dc-fb38ca847e3e None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:24.851183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ed7f4-94e5-47a3-93dd-62f5e7664f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:24.851183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ed7f4-94e5-47a3-93dd-62f5e7664f9e None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:24.851183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ed7f4-94e5-47a3-93dd-62f5e7664f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:24.851183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ed7f4-94e5-47a3-93dd-62f5e7664f9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:24.860827 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ed7f4-94e5-47a3-93dd-62f5e7664f9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:53:24.868836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-aa2ed7f4-94e5-47a3-93dd-62f5e7664f9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:53:24.868836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:24.868836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:24.868836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:24.868836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:24.868836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:24.868836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:24.868836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:24.868836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:24.868836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:24.868836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:24.868836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:24.868836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:53:24.868836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:53:24.868836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:53:24.868836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:53:24.870050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:53:24.870050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:53:24.870050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:53:24.870050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:53:24.870050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:24.870050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:24.870050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:24.870050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:24.870050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:24.870050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:24.870050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:24.870050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:24.870050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:24.870050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:24.870050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:24.870050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:24.871406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:24.871406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:24.871406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:24.871406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:24.871406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:53:24.871406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:53:24.871406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:53:24.871406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:53:24.871406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:53:24.871406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:53:24.871406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:53:24.871406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:24.871406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 605/1208] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:53:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:53:24.877185 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace1633-b3d9-4ffe-8e97-1f3ca5da29b0 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:24.920957 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:24.984946 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace1633-b3d9-4ffe-8e97-1f3ca5da29b0 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:25.019020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e2e82-3761-4303-96b9-dd6417e90d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:25.019696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e2e82-3761-4303-96b9-dd6417e90d57 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:25.019986 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e2e82-3761-4303-96b9-dd6417e90d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:25.020257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e2e82-3761-4303-96b9-dd6417e90d57 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:25.021055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e2e82-3761-4303-96b9-dd6417e90d57 None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:25.021357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e2e82-3761-4303-96b9-dd6417e90d57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:25.022724 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e2e82-3761-4303-96b9-dd6417e90d57 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:25.032243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 604/1209] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:53:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:25.038259 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ed7f4-94e5-47a3-93dd-62f5e7664f9e None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:25.089833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:25.144876 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ed7f4-94e5-47a3-93dd-62f5e7664f9e None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:25.178646 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4e498-91cd-4b66-93de-70c3a8831b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:25.179461 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4e498-91cd-4b66-93de-70c3a8831b96 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:25.179740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4e498-91cd-4b66-93de-70c3a8831b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:25.180016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4e498-91cd-4b66-93de-70c3a8831b96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:25.191354 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4e498-91cd-4b66-93de-70c3a8831b96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:25.192833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4e498-91cd-4b66-93de-70c3a8831b96 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:25.210010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-41e4e498-91cd-4b66-93de-70c3a8831b96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:25.210010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:25.210010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:25.210010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:25.210010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:25.210010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:25.210010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:25.210010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:25.210010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:25.210010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:25.210010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:25.210010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:25.210010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:53:25.210010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:53:25.210010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:53:25.211428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:53:25.211428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:25.211428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:25.211428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:25.211428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:25.211428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:25.211428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:25.211428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:25.211428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:25.211428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:25.211428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:25.211428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:25.211428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:25.211428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:25.211428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:25.211428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:25.211428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:25.212494 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:53:25.212494 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:53:25.212494 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:25.212494 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:25.212494 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 606/1210] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:25.217670 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e2e82-3761-4303-96b9-dd6417e90d57 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:25.319486 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:25.322465 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e2e82-3761-4303-96b9-dd6417e90d57 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:25.355824 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed147e1-0a3e-4e7a-98f5-70c7f346fc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:25.356483 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed147e1-0a3e-4e7a-98f5-70c7f346fc6e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:25.356762 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed147e1-0a3e-4e7a-98f5-70c7f346fc6e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:25.357041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed147e1-0a3e-4e7a-98f5-70c7f346fc6e None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:25.357762 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed147e1-0a3e-4e7a-98f5-70c7f346fc6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:25.358063 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed147e1-0a3e-4e7a-98f5-70c7f346fc6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:25.360176 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed147e1-0a3e-4e7a-98f5-70c7f346fc6e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:25.372567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 605/1211] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:25.378692 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4e498-91cd-4b66-93de-70c3a8831b96 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:25.481799 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4e498-91cd-4b66-93de-70c3a8831b96 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:25.483311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:25.518356 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18855730-313d-4333-8a96-a03727e30fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:25.519187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18855730-313d-4333-8a96-a03727e30fbe None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:25.519467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18855730-313d-4333-8a96-a03727e30fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:25.519741 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18855730-313d-4333-8a96-a03727e30fbe None admin] PATH_INFO: `/v3/users/placement` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:25.526795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18855730-313d-4333-8a96-a03727e30fbe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:25.528252 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18855730-313d-4333-8a96-a03727e30fbe None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:25.543219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-18855730-313d-4333-8a96-a03727e30fbe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 16 14:53:25.543219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:25.543219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:25.543219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:25.543219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:25.543219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:25.543219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:25.543219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:25.543219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:25.543219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:25.543219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:25.543219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:25.543219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:53:25.543219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:53:25.543219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:53:25.543219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:53:25.544443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:25.544443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:25.544443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:53:25.544443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:53:25.544443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:53:25.544443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:53:25.544443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:25.544443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:25.544443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:25.544443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:25.544443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:25.544443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:25.544443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:25.544443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:25.544443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:25.544443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:25.544443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:25.544443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:25.545782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:25.545782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:25.545782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:53:25.545782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:53:25.545782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:53:25.545782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:53:25.545782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:53:25.545782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:53:25.545782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 16 14:53:25.545782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:25.545782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 607/1212] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:53:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:25.551098 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed147e1-0a3e-4e7a-98f5-70c7f346fc6e None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:25.692621 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed147e1-0a3e-4e7a-98f5-70c7f346fc6e None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:25.728300 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a7277-8353-4946-9e36-b390a1a46c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:25.728963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a7277-8353-4946-9e36-b390a1a46c29 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:25.729239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a7277-8353-4946-9e36-b390a1a46c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:25.729507 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a7277-8353-4946-9e36-b390a1a46c29 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:25.730335 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a7277-8353-4946-9e36-b390a1a46c29 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:25.730634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a7277-8353-4946-9e36-b390a1a46c29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:25.732176 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a7277-8353-4946-9e36-b390a1a46c29 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:25.750328 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 606/1213] 10.222.0.27 () {56 vars in 1143 bytes} [Sat Jul 16 14:53:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:53:25.755895 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18855730-313d-4333-8a96-a03727e30fbe None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:25.855379 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18855730-313d-4333-8a96-a03727e30fbe None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:25.892303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb8521-bc50-4b8d-b690-241d05d77d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:25.892968 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb8521-bc50-4b8d-b690-241d05d77d42 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:25.893251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb8521-bc50-4b8d-b690-241d05d77d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:25.893527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb8521-bc50-4b8d-b690-241d05d77d42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:25.926454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:25.964715 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fb8521-bc50-4b8d-b690-241d05d77d42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:25.966188 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fb8521-bc50-4b8d-b690-241d05d77d42 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:26.064272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 608/1214] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:53:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 16 14:53:26.070217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a7277-8353-4946-9e36-b390a1a46c29 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:26.093748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:26.178053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a7277-8353-4946-9e36-b390a1a46c29 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:26.212239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29119384-6624-4eae-8f2f-06888340a2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:26.213090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29119384-6624-4eae-8f2f-06888340a2bb None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:26.213376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29119384-6624-4eae-8f2f-06888340a2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:26.213645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29119384-6624-4eae-8f2f-06888340a2bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:26.225002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29119384-6624-4eae-8f2f-06888340a2bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:26.226494 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29119384-6624-4eae-8f2f-06888340a2bb None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:26.245004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-29119384-6624-4eae-8f2f-06888340a2bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:26.245004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:26.245004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:26.245004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:26.245004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:26.245004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:26.245004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:26.245004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:26.245004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:26.245004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:26.245004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:26.245004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:26.245004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:53:26.245004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:53:26.245004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:53:26.246119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:53:26.246119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:26.246119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:26.246119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:26.246119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:26.246119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:26.246119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:26.246119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:26.246119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:26.246119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:26.246119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:26.246119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:26.246119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:26.246119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:26.246119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:26.246119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:26.246119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:26.247266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:53:26.247266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:53:26.247266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:26.247266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:53:26.247266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 607/1215] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:53:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:26.252256 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb8521-bc50-4b8d-b690-241d05d77d42 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:26.313018 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:26.351815 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb8521-bc50-4b8d-b690-241d05d77d42 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:26.386275 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f745ab-628f-4d72-9ce8-6ab6de3c796f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:26.386937 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f745ab-628f-4d72-9ce8-6ab6de3c796f None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:26.387230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f745ab-628f-4d72-9ce8-6ab6de3c796f None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:26.387506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f745ab-628f-4d72-9ce8-6ab6de3c796f None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:26.388223 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f745ab-628f-4d72-9ce8-6ab6de3c796f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:26.388529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f745ab-628f-4d72-9ce8-6ab6de3c796f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:26.389998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f745ab-628f-4d72-9ce8-6ab6de3c796f None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:26.401965 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 609/1216] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:26.408500 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29119384-6624-4eae-8f2f-06888340a2bb None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:26.481806 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:26.511726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29119384-6624-4eae-8f2f-06888340a2bb None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:26.547049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8cdc2e-91aa-4e79-825e-eb6b45c1fd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:26.547902 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cdc2e-91aa-4e79-825e-eb6b45c1fd8b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:26.548190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cdc2e-91aa-4e79-825e-eb6b45c1fd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:26.548466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cdc2e-91aa-4e79-825e-eb6b45c1fd8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:26.555347 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8cdc2e-91aa-4e79-825e-eb6b45c1fd8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:26.556794 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8cdc2e-91aa-4e79-825e-eb6b45c1fd8b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:26.570474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-1d8cdc2e-91aa-4e79-825e-eb6b45c1fd8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:53:26.570474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:26.570474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:26.570474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:26.570474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:26.570474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:26.570474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:26.570474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:26.570474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:26.570474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:26.570474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:26.570474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:26.570474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:53:26.570474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:53:26.570474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:53:26.571687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:53:26.571687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:26.571687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:26.571687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:26.571687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:26.571687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:26.571687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:26.571687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:26.571687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:26.571687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:26.571687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:26.571687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:26.571687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:26.571687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:26.571687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:26.571687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:26.571687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:26.572571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:53:26.572571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:53:26.572571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:53:26.572571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:53:26.572571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:53:26.572571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:53:26.572571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:53:26.572571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:53:26.572571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 608/1217] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:53:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:26.578317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f745ab-628f-4d72-9ce8-6ab6de3c796f None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:26.681395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f745ab-628f-4d72-9ce8-6ab6de3c796f None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:26.704327 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:26.715839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28401b9c-eccd-428c-b4c6-48bfe8f3eeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:26.716495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28401b9c-eccd-428c-b4c6-48bfe8f3eeea None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:26.716770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28401b9c-eccd-428c-b4c6-48bfe8f3eeea None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:26.717039 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28401b9c-eccd-428c-b4c6-48bfe8f3eeea None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:26.717822 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28401b9c-eccd-428c-b4c6-48bfe8f3eeea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:26.718721 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28401b9c-eccd-428c-b4c6-48bfe8f3eeea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:26.720183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28401b9c-eccd-428c-b4c6-48bfe8f3eeea None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:26.732081 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 610/1218] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:53:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:26.738212 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8cdc2e-91aa-4e79-825e-eb6b45c1fd8b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:26.840397 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8cdc2e-91aa-4e79-825e-eb6b45c1fd8b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:26.869456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:26.876845 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086af92d-296b-4150-beb1-816f796ef46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:26.877929 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086af92d-296b-4150-beb1-816f796ef46e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:26.878583 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086af92d-296b-4150-beb1-816f796ef46e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:26.878850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086af92d-296b-4150-beb1-816f796ef46e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:26.879895 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086af92d-296b-4150-beb1-816f796ef46e None admin] RBAC: Adding query filter params (role.id=21119ebd056740048b9ec639ebb20446, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=3a30cee766814c8685578819261705ba) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:26.880558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086af92d-296b-4150-beb1-816f796ef46e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:26.882485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086af92d-296b-4150-beb1-816f796ef46e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:26.891630 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 609/1219] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:53:26 2022] GET /identity/v3/role_assignments?user.id=3a30cee766814c8685578819261705ba&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=21119ebd056740048b9ec639ebb20446 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:28.927917 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01421a8b-194e-4bf8-968e-e44a8d7c7935 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:28.928486 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01421a8b-194e-4bf8-968e-e44a8d7c7935 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:28.928585 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01421a8b-194e-4bf8-968e-e44a8d7c7935 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:28.929977 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 611/1220] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:53:28.936187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086ec91-e3d2-452f-927f-bdaae797f20d None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:28.943489 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086ec91-e3d2-452f-927f-bdaae797f20d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:28.943489 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086ec91-e3d2-452f-927f-bdaae797f20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:28.956731 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-2086ec91-e3d2-452f-927f-bdaae797f20d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:53:28.957982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2086ec91-e3d2-452f-927f-bdaae797f20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:53:29.033667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 610/1221] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:53:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 16 14:53:29.056045 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01421a8b-194e-4bf8-968e-e44a8d7c7935 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:29.164305 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01421a8b-194e-4bf8-968e-e44a8d7c7935 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:29.200800 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51928b6b-55a2-4a2d-babf-f8cdca52d096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:29.201644 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51928b6b-55a2-4a2d-babf-f8cdca52d096 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:29.201927 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51928b6b-55a2-4a2d-babf-f8cdca52d096 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:29.202938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51928b6b-55a2-4a2d-babf-f8cdca52d096 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:29.213835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51928b6b-55a2-4a2d-babf-f8cdca52d096 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:53:29.221938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-51928b6b-55a2-4a2d-babf-f8cdca52d096 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:53:29.221938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:29.221938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:29.221938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:29.221938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:29.221938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:29.221938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:29.221938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:29.221938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:29.221938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:29.221938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:29.221938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:29.221938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:53:29.221938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:53:29.221938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:53:29.221938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:53:29.223201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:53:29.223201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:53:29.223201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:53:29.223201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:53:29.223201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:29.223201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:29.223201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:29.223201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:29.223201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:29.223201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:29.223201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:29.223201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:29.223201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:29.223201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:29.223201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:29.223201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:29.229476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:29.229476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:29.229476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:29.229476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:29.229476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:53:29.229476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:53:29.229476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:53:29.229476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:53:29.229476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:53:29.229476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:53:29.229476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:53:29.229476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:29.229476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 612/1222] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:53:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:53:29.230372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086ec91-e3d2-452f-927f-bdaae797f20d None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:29.335951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086ec91-e3d2-452f-927f-bdaae797f20d None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:29.362364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:29.373902 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e6eda-7c28-47f2-9701-6abd6336cd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:29.375176 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e6eda-7c28-47f2-9701-6abd6336cd24 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:29.375458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e6eda-7c28-47f2-9701-6abd6336cd24 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:29.375732 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e6eda-7c28-47f2-9701-6abd6336cd24 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:29.376526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e6eda-7c28-47f2-9701-6abd6336cd24 None admin] RBAC: Adding query filter params (name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:29.376834 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e6eda-7c28-47f2-9701-6abd6336cd24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:29.378263 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e6eda-7c28-47f2-9701-6abd6336cd24 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:29.387219 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 611/1223] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:53:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:53:29.393955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51928b6b-55a2-4a2d-babf-f8cdca52d096 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:29.442568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:29.504607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51928b6b-55a2-4a2d-babf-f8cdca52d096 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:29.540435 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13398d89-99cf-419a-abb9-f2fd7f1e2172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:29.541239 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13398d89-99cf-419a-abb9-f2fd7f1e2172 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:29.541515 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13398d89-99cf-419a-abb9-f2fd7f1e2172 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:29.541782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13398d89-99cf-419a-abb9-f2fd7f1e2172 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:29.553987 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13398d89-99cf-419a-abb9-f2fd7f1e2172 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:29.561125 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13398d89-99cf-419a-abb9-f2fd7f1e2172 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:29.573282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-13398d89-99cf-419a-abb9-f2fd7f1e2172 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:29.573282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:29.573282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:29.573282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:29.573282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:29.573282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:29.573282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:29.573282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:29.573282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:29.573282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:29.573282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:29.573282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:29.573282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:53:29.573282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:53:29.573282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:53:29.574371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:53:29.574371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:29.574371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:29.574371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:29.574371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:29.574371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:29.574371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:29.574371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:29.574371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:29.574371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:29.574371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:29.574371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:29.574371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:29.574371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:29.574371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:29.574371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:29.574371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:29.575481 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:53:29.575481 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:53:29.575481 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:29.575481 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:29.575481 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 613/1224] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:53:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:53:29.581095 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e6eda-7c28-47f2-9701-6abd6336cd24 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:29.684825 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e6eda-7c28-47f2-9701-6abd6336cd24 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:29.718939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cb50b-628a-4971-a562-84dd104f38db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:29.719532 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cb50b-628a-4971-a562-84dd104f38db None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:29.719828 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cb50b-628a-4971-a562-84dd104f38db None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:29.720083 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cb50b-628a-4971-a562-84dd104f38db None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:29.720794 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cb50b-628a-4971-a562-84dd104f38db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:29.721095 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cb50b-628a-4971-a562-84dd104f38db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:29.722456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cb50b-628a-4971-a562-84dd104f38db None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:29.734548 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 612/1225] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:29.740396 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13398d89-99cf-419a-abb9-f2fd7f1e2172 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:29.837435 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:29.844381 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13398d89-99cf-419a-abb9-f2fd7f1e2172 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:29.878395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99504d-4806-41a1-bc16-900fdb232aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:29.879246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99504d-4806-41a1-bc16-900fdb232aad None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:29.879529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99504d-4806-41a1-bc16-900fdb232aad None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:29.879829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99504d-4806-41a1-bc16-900fdb232aad None admin] PATH_INFO: `/v3/users/placement` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:29.888300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99504d-4806-41a1-bc16-900fdb232aad None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:29.889784 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99504d-4806-41a1-bc16-900fdb232aad None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:29.906412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-8a99504d-4806-41a1-bc16-900fdb232aad None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 16 14:53:29.906412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:29.906412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:29.906412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:29.906412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:29.906412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:29.906412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:29.906412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:29.906412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:29.906412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:29.906412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:29.906412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:29.906412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:53:29.906412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:53:29.906412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:53:29.906412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:53:29.907540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:29.907540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:29.907540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:53:29.907540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:53:29.907540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:53:29.907540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:53:29.907540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:29.907540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:29.907540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:29.907540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:29.907540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:29.907540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:29.907540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:29.907540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:29.907540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:29.907540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:29.907540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:29.907540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:29.908650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:29.908650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:29.908650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:53:29.908650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:53:29.908650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:53:29.908650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:53:29.908650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:53:29.908650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:53:29.908650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 16 14:53:29.908650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:29.908650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 614/1226] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:53:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:29.914614 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cb50b-628a-4971-a562-84dd104f38db None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:29.963735 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:30.018007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cb50b-628a-4971-a562-84dd104f38db None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:30.054552 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ac6c4-7997-4ac0-9559-0959464b9d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:30.055251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ac6c4-7997-4ac0-9559-0959464b9d18 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:30.055531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ac6c4-7997-4ac0-9559-0959464b9d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:30.055806 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ac6c4-7997-4ac0-9559-0959464b9d18 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:30.056646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ac6c4-7997-4ac0-9559-0959464b9d18 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:30.056959 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ac6c4-7997-4ac0-9559-0959464b9d18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:30.058376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ac6c4-7997-4ac0-9559-0959464b9d18 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:30.077056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 613/1227] 10.222.0.27 () {56 vars in 1143 bytes} [Sat Jul 16 14:53:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:30.083953 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99504d-4806-41a1-bc16-900fdb232aad None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:30.185604 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99504d-4806-41a1-bc16-900fdb232aad None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:30.220524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6aa24-674a-459f-9a77-f479a8a9ecbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:30.221306 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6aa24-674a-459f-9a77-f479a8a9ecbf None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:30.221604 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6aa24-674a-459f-9a77-f479a8a9ecbf None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:30.221874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6aa24-674a-459f-9a77-f479a8a9ecbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:30.231838 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6aa24-674a-459f-9a77-f479a8a9ecbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:30.233271 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6aa24-674a-459f-9a77-f479a8a9ecbf None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:30.252231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-84e6aa24-674a-459f-9a77-f479a8a9ecbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:30.252231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:30.252231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:30.252231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:30.252231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:30.252231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:30.252231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:30.252231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:30.252231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:30.252231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:30.252231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:30.252231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:30.252231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:53:30.252231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:53:30.252231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:53:30.253319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:53:30.253319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:30.253319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:30.253319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:30.253319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:30.253319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:30.253319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:30.253319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:30.253319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:30.253319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:30.253319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:30.253319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:30.253319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:30.253319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:30.253319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:30.253319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:30.253319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:30.254696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:53:30.254696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:53:30.254696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:30.254696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:30.254696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 615/1228] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:53:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:30.259651 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ac6c4-7997-4ac0-9559-0959464b9d18 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:30.360440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ac6c4-7997-4ac0-9559-0959464b9d18 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:30.363089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:30.398487 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4dcc02-a2af-4ee1-9c4e-c55a3ca443bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:30.399180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4dcc02-a2af-4ee1-9c4e-c55a3ca443bf None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:30.399462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4dcc02-a2af-4ee1-9c4e-c55a3ca443bf None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:30.399732 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4dcc02-a2af-4ee1-9c4e-c55a3ca443bf None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:30.400551 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4dcc02-a2af-4ee1-9c4e-c55a3ca443bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:30.400850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4dcc02-a2af-4ee1-9c4e-c55a3ca443bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:30.402209 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4dcc02-a2af-4ee1-9c4e-c55a3ca443bf None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:30.413652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 614/1229] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:30.419425 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6aa24-674a-459f-9a77-f479a8a9ecbf None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:30.429509 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:30.523844 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6aa24-674a-459f-9a77-f479a8a9ecbf None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:30.558020 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e0a02-7d12-4609-b7b5-0896a385a6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:30.558798 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e0a02-7d12-4609-b7b5-0896a385a6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:30.559100 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e0a02-7d12-4609-b7b5-0896a385a6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:30.559374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e0a02-7d12-4609-b7b5-0896a385a6c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:30.567067 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e0a02-7d12-4609-b7b5-0896a385a6c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:30.568559 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e0a02-7d12-4609-b7b5-0896a385a6c7 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:30.582862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-474e0a02-7d12-4609-b7b5-0896a385a6c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:53:30.582862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:30.582862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:30.582862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:30.582862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:30.582862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:30.582862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:30.582862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:30.582862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:30.582862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:30.582862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:30.582862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:30.582862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:53:30.582862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:53:30.582862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:53:30.584005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:53:30.584005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:30.584005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:30.584005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:30.584005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:30.584005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:30.584005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:30.584005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:30.584005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:30.584005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:30.584005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:30.584005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:30.584005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:30.584005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:30.584005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:30.584005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:30.584005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:30.585397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:53:30.585397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:53:30.585397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:53:30.585397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:53:30.585397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:53:30.585397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:53:30.585397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:53:30.585397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:30.585397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 616/1230] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:53:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:53:30.590225 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4dcc02-a2af-4ee1-9c4e-c55a3ca443bf None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:30.692099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4dcc02-a2af-4ee1-9c4e-c55a3ca443bf None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:30.726760 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27aeeb-12b2-4503-9d09-06ff8cb64020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:30.727522 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27aeeb-12b2-4503-9d09-06ff8cb64020 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:30.727802 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27aeeb-12b2-4503-9d09-06ff8cb64020 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:30.728075 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27aeeb-12b2-4503-9d09-06ff8cb64020 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:30.728874 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27aeeb-12b2-4503-9d09-06ff8cb64020 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:30.729179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27aeeb-12b2-4503-9d09-06ff8cb64020 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:30.730616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27aeeb-12b2-4503-9d09-06ff8cb64020 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:30.744114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 615/1231] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:53:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:30.750639 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e0a02-7d12-4609-b7b5-0896a385a6c7 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:30.817260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:30.851919 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e0a02-7d12-4609-b7b5-0896a385a6c7 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:30.889500 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011e9b6-f803-4442-b76b-a12073871382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:30.891054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011e9b6-f803-4442-b76b-a12073871382 None admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:30.891341 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011e9b6-f803-4442-b76b-a12073871382 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:30.891681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011e9b6-f803-4442-b76b-a12073871382 None admin] PATH_INFO: `/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/3a30cee766814c8685578819261705ba/roles/21119ebd056740048b9ec639ebb20446` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:30.911029 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011e9b6-f803-4442-b76b-a12073871382 None admin] RBAC: Authorizing `identity:create_grant(project_id=2739f00eb01e46a9acf90ffa360d7d8c, user_id=3a30cee766814c8685578819261705ba, role_id=21119ebd056740048b9ec639ebb20446)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:30.912500 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011e9b6-f803-4442-b76b-a12073871382 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:31.014330 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 617/1232] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:53:30 2022] PUT /identity/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/3a30cee766814c8685578819261705ba/roles/21119ebd056740048b9ec639ebb20446 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:53:33.065601 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7400f-f825-4fba-b2e3-9339f50e3369 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:33.066124 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7400f-f825-4fba-b2e3-9339f50e3369 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:33.066124 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7400f-f825-4fba-b2e3-9339f50e3369 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:33.067650 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 616/1233] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:53:33.076740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97346d-7f9c-4964-bcb6-6089c8268ca0 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:33.077393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97346d-7f9c-4964-bcb6-6089c8268ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:33.077393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97346d-7f9c-4964-bcb6-6089c8268ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:33.095102 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-ef97346d-7f9c-4964-bcb6-6089c8268ca0 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:53:33.096311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ef97346d-7f9c-4964-bcb6-6089c8268ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:53:33.173789 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 618/1234] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:53:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:53:33.196973 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7400f-f825-4fba-b2e3-9339f50e3369 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:33.224498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:33.305779 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7400f-f825-4fba-b2e3-9339f50e3369 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:33.345951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57656580-ac6e-49f9-81f3-4857ce818582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:33.346763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57656580-ac6e-49f9-81f3-4857ce818582 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:33.347080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57656580-ac6e-49f9-81f3-4857ce818582 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:33.347357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57656580-ac6e-49f9-81f3-4857ce818582 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:33.359897 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57656580-ac6e-49f9-81f3-4857ce818582 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:53:33.367998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-57656580-ac6e-49f9-81f3-4857ce818582 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:53:33.367998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:33.367998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:33.367998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:33.367998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:33.367998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:33.367998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:33.367998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:33.367998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:33.367998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:33.367998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:33.367998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:33.367998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:53:33.367998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:53:33.367998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:53:33.367998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:53:33.369179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:53:33.369179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:53:33.369179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:53:33.369179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:53:33.369179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:33.369179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:33.369179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:33.369179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:33.369179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:33.369179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:33.369179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:33.369179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:33.369179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:33.369179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:33.369179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:33.369179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:33.370410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:33.370410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:33.370410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:33.370410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:33.370410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:53:33.370410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:53:33.370410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:53:33.370410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:53:33.370410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:53:33.370410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:53:33.370410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 16 14:53:33.370410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:53:33.370410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 617/1235] 10.222.0.27 () {56 vars in 1102 bytes} [Sat Jul 16 14:53:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:33.376128 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97346d-7f9c-4964-bcb6-6089c8268ca0 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:33.427373 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:33.481949 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97346d-7f9c-4964-bcb6-6089c8268ca0 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:33.517489 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7543e-ad6b-48fa-9bb7-219978054087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:33.518160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7543e-ad6b-48fa-9bb7-219978054087 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:33.518439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7543e-ad6b-48fa-9bb7-219978054087 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:33.518718 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7543e-ad6b-48fa-9bb7-219978054087 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:33.519573 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7543e-ad6b-48fa-9bb7-219978054087 None admin] RBAC: Adding query filter params (name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:33.519886 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7543e-ad6b-48fa-9bb7-219978054087 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:33.521253 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7543e-ad6b-48fa-9bb7-219978054087 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:33.529756 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 619/1236] 10.222.0.27 () {56 vars in 1103 bytes} [Sat Jul 16 14:53:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:33.536277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57656580-ac6e-49f9-81f3-4857ce818582 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:33.621559 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:33.637417 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57656580-ac6e-49f9-81f3-4857ce818582 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:33.672334 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09eeef-ee08-4ae1-975a-810dc09667fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:33.673130 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09eeef-ee08-4ae1-975a-810dc09667fe None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:33.673406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09eeef-ee08-4ae1-975a-810dc09667fe None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:33.673671 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09eeef-ee08-4ae1-975a-810dc09667fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:33.685259 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09eeef-ee08-4ae1-975a-810dc09667fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:33.687447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09eeef-ee08-4ae1-975a-810dc09667fe None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:33.706850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-db09eeef-ee08-4ae1-975a-810dc09667fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:33.706850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:33.706850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:33.706850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:33.706850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:33.706850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:33.706850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:33.706850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:33.706850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:33.706850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:33.706850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:33.706850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:33.706850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:53:33.706850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:53:33.706850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:53:33.712756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:53:33.712756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:33.712756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:33.712756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:33.712756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:33.712756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:33.712756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:33.712756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:33.712756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:33.712756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:33.712756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:33.712756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:33.712756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:33.712756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:33.712756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:33.712756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:33.712756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:33.713838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:53:33.713838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:53:33.713838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:33.713838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:53:33.713838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 618/1237] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:53:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:33.714929 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7543e-ad6b-48fa-9bb7-219978054087 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:33.817538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7543e-ad6b-48fa-9bb7-219978054087 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:33.826663 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:33.852279 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf92710-3bd5-4985-ac8e-2dcbde806387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:33.852938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf92710-3bd5-4985-ac8e-2dcbde806387 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:33.853213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf92710-3bd5-4985-ac8e-2dcbde806387 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:33.853481 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf92710-3bd5-4985-ac8e-2dcbde806387 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:33.854237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf92710-3bd5-4985-ac8e-2dcbde806387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:33.854520 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf92710-3bd5-4985-ac8e-2dcbde806387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:33.855901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf92710-3bd5-4985-ac8e-2dcbde806387 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:33.867471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 620/1238] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:33.873296 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09eeef-ee08-4ae1-975a-810dc09667fe None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:33.975476 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09eeef-ee08-4ae1-975a-810dc09667fe None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:34.012350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14834c6-3a2a-492a-a783-1b59e4ebfd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:34.013312 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14834c6-3a2a-492a-a783-1b59e4ebfd09 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:34.013595 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14834c6-3a2a-492a-a783-1b59e4ebfd09 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:34.013863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14834c6-3a2a-492a-a783-1b59e4ebfd09 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:34.017015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:34.022398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14834c6-3a2a-492a-a783-1b59e4ebfd09 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:34.029317 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14834c6-3a2a-492a-a783-1b59e4ebfd09 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:34.038530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-d14834c6-3a2a-492a-a783-1b59e4ebfd09 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 16 14:53:34.038530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:34.038530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:34.038530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:34.038530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:34.038530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:34.038530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:34.038530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:34.038530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:34.038530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:34.038530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:34.038530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:34.038530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:53:34.038530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:53:34.038530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:53:34.038530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:53:34.039682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:34.039682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:34.039682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:53:34.039682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:53:34.039682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:53:34.039682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:53:34.039682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:34.039682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:34.039682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:34.039682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:34.039682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:34.039682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:34.039682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:34.039682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:34.039682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:34.039682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:34.039682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:34.039682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:34.040914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:34.040914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:34.040914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:53:34.040914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:53:34.040914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:53:34.040914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:53:34.040914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:53:34.040914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:53:34.040914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 16 14:53:34.040914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:53:34.040914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 619/1239] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:53:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:53:34.046232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf92710-3bd5-4985-ac8e-2dcbde806387 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:34.148984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf92710-3bd5-4985-ac8e-2dcbde806387 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:34.182836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afa9b4-6751-44af-837a-aed939c1f0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:34.183533 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afa9b4-6751-44af-837a-aed939c1f0af None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:34.183816 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afa9b4-6751-44af-837a-aed939c1f0af None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:34.184111 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afa9b4-6751-44af-837a-aed939c1f0af None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:34.184943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5afa9b4-6751-44af-837a-aed939c1f0af None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:34.185248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5afa9b4-6751-44af-837a-aed939c1f0af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:34.187385 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5afa9b4-6751-44af-837a-aed939c1f0af None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:34.205714 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 621/1240] 10.222.0.27 () {56 vars in 1143 bytes} [Sat Jul 16 14:53:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:34.211896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14834c6-3a2a-492a-a783-1b59e4ebfd09 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:34.315045 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14834c6-3a2a-492a-a783-1b59e4ebfd09 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:34.349332 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8db763-dff2-4547-82fc-ef6401fbe484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:34.349969 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8db763-dff2-4547-82fc-ef6401fbe484 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:34.350245 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8db763-dff2-4547-82fc-ef6401fbe484 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:34.350512 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8db763-dff2-4547-82fc-ef6401fbe484 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:34.412284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:34.418713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8db763-dff2-4547-82fc-ef6401fbe484 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:34.420286 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8db763-dff2-4547-82fc-ef6401fbe484 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:34.523301 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 620/1241] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:53:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 14:53:34.529380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afa9b4-6751-44af-837a-aed939c1f0af None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:34.557957 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:34.629717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afa9b4-6751-44af-837a-aed939c1f0af None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:34.664322 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7eeb4-7f9b-4eb3-81c0-e81ce0a5b5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:34.665101 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7eeb4-7f9b-4eb3-81c0-e81ce0a5b5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:34.665382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7eeb4-7f9b-4eb3-81c0-e81ce0a5b5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:34.665653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7eeb4-7f9b-4eb3-81c0-e81ce0a5b5f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:34.676488 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7eeb4-7f9b-4eb3-81c0-e81ce0a5b5f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:34.677935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7eeb4-7f9b-4eb3-81c0-e81ce0a5b5f6 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:34.695824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-38c7eeb4-7f9b-4eb3-81c0-e81ce0a5b5f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:34.695824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:34.695824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:34.695824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:34.695824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:34.695824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:34.695824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:34.695824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:34.695824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:34.695824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:34.695824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:34.695824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:34.695824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:53:34.695824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:53:34.695824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:53:34.696924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:53:34.696924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:34.696924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:34.696924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:34.696924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:34.696924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:34.696924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:34.696924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:34.696924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:34.696924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:34.696924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:34.696924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:34.696924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:34.696924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:34.696924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:34.696924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:34.696924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:34.698236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:53:34.698236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:53:34.698236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:34.698236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:34.698236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 622/1242] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:53:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:34.704012 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8db763-dff2-4547-82fc-ef6401fbe484 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:34.799652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:34.805492 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8db763-dff2-4547-82fc-ef6401fbe484 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:34.840402 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1de9c0-687e-4c7e-8b1e-44e46b756665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:34.841084 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1de9c0-687e-4c7e-8b1e-44e46b756665 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:34.841363 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1de9c0-687e-4c7e-8b1e-44e46b756665 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:34.841632 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1de9c0-687e-4c7e-8b1e-44e46b756665 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:34.842431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1de9c0-687e-4c7e-8b1e-44e46b756665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:34.843872 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1de9c0-687e-4c7e-8b1e-44e46b756665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:34.845242 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1de9c0-687e-4c7e-8b1e-44e46b756665 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:34.856422 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 621/1243] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:34.862230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7eeb4-7f9b-4eb3-81c0-e81ce0a5b5f6 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:34.944882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:34.967455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7eeb4-7f9b-4eb3-81c0-e81ce0a5b5f6 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:35.001368 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bad2d2-6e4f-4976-8595-d6a46f8a6f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:35.002161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bad2d2-6e4f-4976-8595-d6a46f8a6f06 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:35.002446 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bad2d2-6e4f-4976-8595-d6a46f8a6f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:35.002713 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bad2d2-6e4f-4976-8595-d6a46f8a6f06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:35.009296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bad2d2-6e4f-4976-8595-d6a46f8a6f06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:35.010767 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bad2d2-6e4f-4976-8595-d6a46f8a6f06 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:35.026075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-d9bad2d2-6e4f-4976-8595-d6a46f8a6f06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:53:35.026075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:35.026075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:35.026075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:35.026075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:35.026075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:35.026075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:35.026075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:35.026075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:35.026075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:35.026075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:35.026075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:35.026075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:53:35.026075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:53:35.026075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:53:35.027276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:53:35.027276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:35.027276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:35.027276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:35.027276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:35.027276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:35.027276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:35.027276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:35.027276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:35.027276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:35.027276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:35.027276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:35.027276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:35.027276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:35.027276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:35.027276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:35.027276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:35.028482 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:53:35.028482 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:53:35.028482 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:53:35.028482 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:53:35.028482 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:53:35.028482 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:53:35.028482 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:53:35.028482 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:35.028482 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 623/1244] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:53:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:35.034748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1de9c0-687e-4c7e-8b1e-44e46b756665 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:35.144808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1de9c0-687e-4c7e-8b1e-44e46b756665 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:35.180278 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28339af-79ea-41c7-a2f1-ca2af218d395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:35.180863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28339af-79ea-41c7-a2f1-ca2af218d395 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:35.181146 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28339af-79ea-41c7-a2f1-ca2af218d395 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:35.181421 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28339af-79ea-41c7-a2f1-ca2af218d395 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:35.182212 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28339af-79ea-41c7-a2f1-ca2af218d395 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:35.182522 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28339af-79ea-41c7-a2f1-ca2af218d395 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:35.184005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28339af-79ea-41c7-a2f1-ca2af218d395 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:35.196647 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 622/1245] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:53:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:35.203241 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bad2d2-6e4f-4976-8595-d6a46f8a6f06 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:35.305470 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bad2d2-6e4f-4976-8595-d6a46f8a6f06 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:35.339382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267cd4a-3e43-47c7-b8a6-8c3ed3ebd676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:35.340036 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267cd4a-3e43-47c7-b8a6-8c3ed3ebd676 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:35.340320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267cd4a-3e43-47c7-b8a6-8c3ed3ebd676 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:35.340597 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267cd4a-3e43-47c7-b8a6-8c3ed3ebd676 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:35.341436 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267cd4a-3e43-47c7-b8a6-8c3ed3ebd676 None admin] RBAC: Adding query filter params (role.id=21119ebd056740048b9ec639ebb20446, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=3a30cee766814c8685578819261705ba) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:35.341795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267cd4a-3e43-47c7-b8a6-8c3ed3ebd676 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:35.343975 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4267cd4a-3e43-47c7-b8a6-8c3ed3ebd676 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:35.349921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 624/1246] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:53:35 2022] GET /identity/v3/role_assignments?user.id=3a30cee766814c8685578819261705ba&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=21119ebd056740048b9ec639ebb20446 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:53:37.483157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae74ef3-6b6d-400d-a93b-17ef5d19e4e7 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:37.483714 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae74ef3-6b6d-400d-a93b-17ef5d19e4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:37.484092 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae74ef3-6b6d-400d-a93b-17ef5d19e4e7 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:37.485508 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 623/1247] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:53:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:53:37.491101 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f273fc-cbdf-409f-b18e-0d786b476c26 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:37.491601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f273fc-cbdf-409f-b18e-0d786b476c26 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:37.491722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f273fc-cbdf-409f-b18e-0d786b476c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:37.496744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:37.510160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-50f273fc-cbdf-409f-b18e-0d786b476c26 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:53:37.511396 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-50f273fc-cbdf-409f-b18e-0d786b476c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:53:37.579861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 625/1248] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:53:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:53:37.603364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae74ef3-6b6d-400d-a93b-17ef5d19e4e7 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:37.621623 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:37.732754 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae74ef3-6b6d-400d-a93b-17ef5d19e4e7 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:37.770399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9d58b-5cfb-4282-b42b-4558f5c45d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:37.771142 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9d58b-5cfb-4282-b42b-4558f5c45d79 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:37.771466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9d58b-5cfb-4282-b42b-4558f5c45d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:37.771736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9d58b-5cfb-4282-b42b-4558f5c45d79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:37.783079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f9d58b-5cfb-4282-b42b-4558f5c45d79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:53:37.790799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-d4f9d58b-5cfb-4282-b42b-4558f5c45d79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:53:37.790799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:37.790799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:37.790799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:37.790799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:37.790799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:37.790799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:37.790799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:37.790799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:37.790799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:37.790799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:37.790799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:37.790799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:53:37.790799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:53:37.790799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:53:37.790799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:53:37.791829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:53:37.791829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:53:37.791829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:53:37.791829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:53:37.791829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:37.791829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:37.791829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:37.791829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:37.791829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:37.791829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:37.791829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:37.791829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:37.791829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:37.791829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:37.791829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:37.791829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:37.792667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:37.792667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:37.792667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:37.792667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:37.792667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:53:37.792667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:53:37.792667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:53:37.792667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:53:37.792667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:53:37.792667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:53:37.792667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:53:37.792667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:53:37.792667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 624/1249] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:53:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:37.799852 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f273fc-cbdf-409f-b18e-0d786b476c26 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:37.903904 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f273fc-cbdf-409f-b18e-0d786b476c26 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:37.939758 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:37.944831 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace84d3b-dbb1-431e-800d-2287a34cc5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:37.946708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace84d3b-dbb1-431e-800d-2287a34cc5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:37.946708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace84d3b-dbb1-431e-800d-2287a34cc5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:37.946708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace84d3b-dbb1-431e-800d-2287a34cc5e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:37.947058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace84d3b-dbb1-431e-800d-2287a34cc5e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:37.947058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace84d3b-dbb1-431e-800d-2287a34cc5e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:37.948474 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace84d3b-dbb1-431e-800d-2287a34cc5e5 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:37.957428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 626/1250] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:53:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:37.963565 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9d58b-5cfb-4282-b42b-4558f5c45d79 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:38.044185 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:38.067430 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9d58b-5cfb-4282-b42b-4558f5c45d79 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:38.103236 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131e9ed-d66d-46e3-ac6b-14330d28fcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:38.104034 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131e9ed-d66d-46e3-ac6b-14330d28fcdf None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:38.104315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131e9ed-d66d-46e3-ac6b-14330d28fcdf None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:38.104589 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131e9ed-d66d-46e3-ac6b-14330d28fcdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:38.116386 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4131e9ed-d66d-46e3-ac6b-14330d28fcdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:38.117845 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4131e9ed-d66d-46e3-ac6b-14330d28fcdf None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:38.136351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-4131e9ed-d66d-46e3-ac6b-14330d28fcdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:38.136351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:38.136351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:38.136351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:38.136351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:38.136351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:38.136351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:38.136351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:38.136351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:38.136351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:38.136351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:38.136351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:38.136351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:53:38.136351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:53:38.136351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:53:38.137499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:53:38.137499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:38.137499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:38.137499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:38.137499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:38.137499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:38.137499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:38.137499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:38.137499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:38.137499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:38.137499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:38.137499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:38.137499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:38.137499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:38.137499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:38.137499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:38.137499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:38.138795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:53:38.138795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:53:38.138795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:38.138795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:53:38.138795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 625/1251] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:53:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:38.144200 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace84d3b-dbb1-431e-800d-2287a34cc5e5 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:38.251548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace84d3b-dbb1-431e-800d-2287a34cc5e5 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:38.286085 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae375a25-fdbf-400c-aec6-32eb5d448bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:38.286753 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae375a25-fdbf-400c-aec6-32eb5d448bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:38.287057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae375a25-fdbf-400c-aec6-32eb5d448bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:38.287330 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae375a25-fdbf-400c-aec6-32eb5d448bb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:38.288049 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae375a25-fdbf-400c-aec6-32eb5d448bb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:38.288345 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae375a25-fdbf-400c-aec6-32eb5d448bb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:38.289726 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae375a25-fdbf-400c-aec6-32eb5d448bb5 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:38.305095 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 627/1252] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:38.311646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131e9ed-d66d-46e3-ac6b-14330d28fcdf None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:38.412491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131e9ed-d66d-46e3-ac6b-14330d28fcdf None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:38.440425 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:38.448262 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09eca5-b1bb-4eaf-80bc-2312cf57785c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:38.449062 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09eca5-b1bb-4eaf-80bc-2312cf57785c None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:38.449343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09eca5-b1bb-4eaf-80bc-2312cf57785c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:38.449615 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09eca5-b1bb-4eaf-80bc-2312cf57785c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:38.457884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a09eca5-b1bb-4eaf-80bc-2312cf57785c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:38.459359 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a09eca5-b1bb-4eaf-80bc-2312cf57785c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:38.474373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-8a09eca5-b1bb-4eaf-80bc-2312cf57785c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 16 14:53:38.474373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:38.474373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:38.474373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:38.474373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:38.474373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:38.474373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:38.474373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:38.474373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:38.474373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:38.474373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:38.474373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:38.474373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:53:38.474373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:53:38.474373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:53:38.474373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:53:38.477896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:38.477896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:38.477896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:53:38.477896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:53:38.477896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:53:38.477896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:53:38.477896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:38.477896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:38.477896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:38.477896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:38.477896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:38.477896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:38.477896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:38.477896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:38.477896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:38.477896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:38.477896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:38.477896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:38.479181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:38.479181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:38.479181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:53:38.479181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:53:38.479181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:53:38.479181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:53:38.479181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:53:38.479181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:53:38.479181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 16 14:53:38.479181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:53:38.479181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 626/1253] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:53:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:38.481722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae375a25-fdbf-400c-aec6-32eb5d448bb5 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:38.536038 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:38.578165 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae375a25-fdbf-400c-aec6-32eb5d448bb5 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:38.612709 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86280629-3c08-43b8-b638-1b3aaf3f506b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:38.613444 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86280629-3c08-43b8-b638-1b3aaf3f506b None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:38.613901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86280629-3c08-43b8-b638-1b3aaf3f506b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:38.614658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86280629-3c08-43b8-b638-1b3aaf3f506b None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:38.615531 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86280629-3c08-43b8-b638-1b3aaf3f506b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:38.615838 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86280629-3c08-43b8-b638-1b3aaf3f506b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:38.617314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86280629-3c08-43b8-b638-1b3aaf3f506b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:38.634417 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 628/1254] 10.222.0.27 () {56 vars in 1143 bytes} [Sat Jul 16 14:53:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:38.641403 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09eca5-b1bb-4eaf-80bc-2312cf57785c None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:38.743773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09eca5-b1bb-4eaf-80bc-2312cf57785c None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:38.778361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276eda6-ede8-46f7-8f37-067912549fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:38.779050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276eda6-ede8-46f7-8f37-067912549fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:38.779337 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276eda6-ede8-46f7-8f37-067912549fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:38.779612 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276eda6-ede8-46f7-8f37-067912549fe1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:38.839227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:38.851004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276eda6-ede8-46f7-8f37-067912549fe1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:38.852468 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276eda6-ede8-46f7-8f37-067912549fe1 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:38.955533 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 627/1255] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:53:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 16 14:53:38.961642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86280629-3c08-43b8-b638-1b3aaf3f506b None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:39.060557 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86280629-3c08-43b8-b638-1b3aaf3f506b None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:39.074533 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:39.097448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42b363-8322-40d8-8854-eb160f32a5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:39.098122 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42b363-8322-40d8-8854-eb160f32a5fe None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:39.098410 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42b363-8322-40d8-8854-eb160f32a5fe None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:39.098683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42b363-8322-40d8-8854-eb160f32a5fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:39.109860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42b363-8322-40d8-8854-eb160f32a5fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:39.111364 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42b363-8322-40d8-8854-eb160f32a5fe None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:39.129057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-5c42b363-8322-40d8-8854-eb160f32a5fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:39.129057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:39.129057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:39.129057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:39.129057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:39.129057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:39.129057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:39.129057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:39.129057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:39.129057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:39.129057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:39.129057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:39.129057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:53:39.129057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:53:39.129057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:53:39.130169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:53:39.130169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:39.130169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:39.130169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:39.130169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:39.130169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:39.130169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:39.130169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:39.130169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:39.130169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:39.130169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:39.130169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:39.130169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:39.130169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:39.130169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:39.130169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:39.130169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:39.131300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:53:39.131300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:53:39.131300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:39.131300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:39.131300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 629/1256] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:53:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:39.136354 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276eda6-ede8-46f7-8f37-067912549fe1 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:39.226414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:39.238315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276eda6-ede8-46f7-8f37-067912549fe1 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:39.273626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da9111-9a8a-4b58-b839-f2c805fec39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:39.274289 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da9111-9a8a-4b58-b839-f2c805fec39d None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:39.274636 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da9111-9a8a-4b58-b839-f2c805fec39d None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:39.275144 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da9111-9a8a-4b58-b839-f2c805fec39d None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:39.276047 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da9111-9a8a-4b58-b839-f2c805fec39d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:39.276720 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da9111-9a8a-4b58-b839-f2c805fec39d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:39.278087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da9111-9a8a-4b58-b839-f2c805fec39d None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:39.290311 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 628/1257] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:39.296364 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42b363-8322-40d8-8854-eb160f32a5fe None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:39.398268 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42b363-8322-40d8-8854-eb160f32a5fe None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:39.432791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b75ad6-7178-4186-a129-d15fd8251ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:39.433576 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b75ad6-7178-4186-a129-d15fd8251ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:39.433855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b75ad6-7178-4186-a129-d15fd8251ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:39.434127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b75ad6-7178-4186-a129-d15fd8251ed0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:39.440951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b75ad6-7178-4186-a129-d15fd8251ed0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:39.442394 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b75ad6-7178-4186-a129-d15fd8251ed0 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:39.456943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-75b75ad6-7178-4186-a129-d15fd8251ed0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:53:39.456943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:39.456943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:39.456943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:39.456943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:39.456943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:39.456943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:39.456943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:39.456943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:39.456943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:39.456943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:39.456943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:39.456943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:53:39.456943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:53:39.456943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:53:39.462426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:53:39.462426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:39.462426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:39.462426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:39.462426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:39.462426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:39.462426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:39.462426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:39.462426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:39.462426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:39.462426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:39.462426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:39.462426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:39.462426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:39.462426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:39.462426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:39.462426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:39.463683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:53:39.463683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:53:39.463683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:53:39.463683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:53:39.463683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:53:39.463683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:53:39.463683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:53:39.463683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:39.463683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 630/1258] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:53:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:39.464351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da9111-9a8a-4b58-b839-f2c805fec39d None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:39.565348 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da9111-9a8a-4b58-b839-f2c805fec39d None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:39.599848 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556fc26a-bd57-482d-8de4-e2b6f968b760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:39.600517 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fc26a-bd57-482d-8de4-e2b6f968b760 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:39.600832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fc26a-bd57-482d-8de4-e2b6f968b760 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:39.601072 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fc26a-bd57-482d-8de4-e2b6f968b760 None admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:39.601862 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556fc26a-bd57-482d-8de4-e2b6f968b760 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:39.602168 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556fc26a-bd57-482d-8de4-e2b6f968b760 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:39.603623 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556fc26a-bd57-482d-8de4-e2b6f968b760 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:39.616136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 629/1259] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:53:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:53:39.622284 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b75ad6-7178-4186-a129-d15fd8251ed0 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:39.648850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:39.726365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b75ad6-7178-4186-a129-d15fd8251ed0 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:39.763164 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468103d-3d39-4485-95d9-d319f7f1806b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:39.763819 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468103d-3d39-4485-95d9-d319f7f1806b None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:39.764828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468103d-3d39-4485-95d9-d319f7f1806b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:39.765187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468103d-3d39-4485-95d9-d319f7f1806b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:39.766028 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468103d-3d39-4485-95d9-d319f7f1806b None admin] RBAC: Adding query filter params (role.id=0efeff46573247f3b9f1e2dec2968672, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=3a30cee766814c8685578819261705ba) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:39.766333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468103d-3d39-4485-95d9-d319f7f1806b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:39.767809 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468103d-3d39-4485-95d9-d319f7f1806b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:39.773675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 631/1260] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:53:39 2022] GET /identity/v3/role_assignments?user.id=3a30cee766814c8685578819261705ba&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=0efeff46573247f3b9f1e2dec2968672 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:41.802316 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920307b2-79d8-4fef-93a7-c22dfa4b7a76 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:41.802959 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920307b2-79d8-4fef-93a7-c22dfa4b7a76 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:41.802959 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920307b2-79d8-4fef-93a7-c22dfa4b7a76 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:41.804295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 630/1261] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:53:41.810405 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0956032-645c-4ebc-9076-3d8045df20aa None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:41.811747 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0956032-645c-4ebc-9076-3d8045df20aa None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:41.811747 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0956032-645c-4ebc-9076-3d8045df20aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:41.830044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-e0956032-645c-4ebc-9076-3d8045df20aa None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:53:41.831289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e0956032-645c-4ebc-9076-3d8045df20aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:53:41.907931 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 632/1262] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:53:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:53:41.931963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920307b2-79d8-4fef-93a7-c22dfa4b7a76 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:41.948790 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:42.041730 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920307b2-79d8-4fef-93a7-c22dfa4b7a76 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:42.077814 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd772ea0-d9aa-4fce-aad0-994a96eba8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:42.078584 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd772ea0-d9aa-4fce-aad0-994a96eba8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:42.078865 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd772ea0-d9aa-4fce-aad0-994a96eba8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:42.079233 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd772ea0-d9aa-4fce-aad0-994a96eba8d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:42.091132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd772ea0-d9aa-4fce-aad0-994a96eba8d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:53:42.099808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-bd772ea0-d9aa-4fce-aad0-994a96eba8d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:53:42.099808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:42.099808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:42.099808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:42.099808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:42.099808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:42.099808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:42.099808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:42.099808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:42.099808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:42.099808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:42.099808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:42.099808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:53:42.099808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:53:42.099808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:53:42.099808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:53:42.103394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:53:42.103394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:53:42.103394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:53:42.103394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:53:42.103394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:42.103394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:42.103394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:42.103394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:42.103394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:42.103394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:42.103394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:42.103394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:42.103394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:42.103394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:42.103394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:42.103394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:42.104515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:42.104515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:42.104515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:42.104515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:42.104515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:53:42.104515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:53:42.104515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:53:42.104515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:53:42.104515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:53:42.104515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:53:42.104515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:53:42.104515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:53:42.104515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 631/1263] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:53:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:42.107453 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0956032-645c-4ebc-9076-3d8045df20aa None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:42.126747 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:42.206972 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0956032-645c-4ebc-9076-3d8045df20aa None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:42.241278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e0646-63e8-4983-bd15-acda5e8142e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:42.241935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e0646-63e8-4983-bd15-acda5e8142e0 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:42.242210 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e0646-63e8-4983-bd15-acda5e8142e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:42.242479 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e0646-63e8-4983-bd15-acda5e8142e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:42.243245 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e0646-63e8-4983-bd15-acda5e8142e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:42.243545 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e0646-63e8-4983-bd15-acda5e8142e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:42.244903 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e0646-63e8-4983-bd15-acda5e8142e0 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:42.254078 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 633/1264] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:53:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:53:42.259721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd772ea0-d9aa-4fce-aad0-994a96eba8d0 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:42.329682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:42.366955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd772ea0-d9aa-4fce-aad0-994a96eba8d0 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:42.400674 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad0161-b62f-487d-b051-695b7aba4a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:42.401467 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad0161-b62f-487d-b051-695b7aba4a22 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:42.401749 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad0161-b62f-487d-b051-695b7aba4a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:42.402026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad0161-b62f-487d-b051-695b7aba4a22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:42.413473 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad0161-b62f-487d-b051-695b7aba4a22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:42.414941 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad0161-b62f-487d-b051-695b7aba4a22 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:42.432094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-6dad0161-b62f-487d-b051-695b7aba4a22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:42.432094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:42.432094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:42.432094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:42.432094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:42.432094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:42.432094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:42.432094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:42.432094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:42.432094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:42.432094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:42.432094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:42.432094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:53:42.432094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:53:42.432094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:53:42.435275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:53:42.435275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:42.435275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:42.435275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:42.435275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:42.435275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:42.435275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:42.435275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:42.435275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:42.435275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:42.435275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:42.435275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:42.435275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:42.435275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:42.435275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:42.435275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:42.435275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:42.437106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:53:42.437106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:53:42.437106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:42.437106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:53:42.437106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 632/1265] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:53:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:42.439235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e0646-63e8-4983-bd15-acda5e8142e0 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:42.525044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:42.544345 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e0646-63e8-4983-bd15-acda5e8142e0 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:42.577949 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da901fb-158c-45cd-990c-07cd5d9379d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:42.578658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da901fb-158c-45cd-990c-07cd5d9379d5 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:42.579028 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da901fb-158c-45cd-990c-07cd5d9379d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:42.579311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da901fb-158c-45cd-990c-07cd5d9379d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:42.580088 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da901fb-158c-45cd-990c-07cd5d9379d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:42.580344 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da901fb-158c-45cd-990c-07cd5d9379d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:42.581725 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da901fb-158c-45cd-990c-07cd5d9379d5 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:42.593491 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 634/1266] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:42.599403 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad0161-b62f-487d-b051-695b7aba4a22 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:42.704057 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad0161-b62f-487d-b051-695b7aba4a22 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:42.723764 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:42.737443 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf771189-ff5e-483e-a425-c18cec00b776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:42.738245 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf771189-ff5e-483e-a425-c18cec00b776 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:42.738529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf771189-ff5e-483e-a425-c18cec00b776 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:42.738804 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf771189-ff5e-483e-a425-c18cec00b776 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:42.746248 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf771189-ff5e-483e-a425-c18cec00b776 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:42.747713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf771189-ff5e-483e-a425-c18cec00b776 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:42.762728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-cf771189-ff5e-483e-a425-c18cec00b776 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 16 14:53:42.762728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:42.762728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:42.762728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:42.762728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:42.762728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:42.762728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:42.762728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:42.762728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:42.762728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:42.762728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:42.762728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:42.762728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:53:42.762728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:53:42.762728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:53:42.762728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:53:42.772462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:42.772462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:42.772462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:53:42.772462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:53:42.772462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:53:42.772462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:53:42.772462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:42.772462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:42.772462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:42.772462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:42.772462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:42.772462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:42.772462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:42.772462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:42.772462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:42.772462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:42.772462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:42.772462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:42.773640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:42.773640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:42.773640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:53:42.773640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:53:42.773640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:53:42.773640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:53:42.773640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:53:42.773640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:53:42.773640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 16 14:53:42.773640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:53:42.773640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 633/1267] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:53:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:42.774411 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da901fb-158c-45cd-990c-07cd5d9379d5 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:42.871835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da901fb-158c-45cd-990c-07cd5d9379d5 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:42.911114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3c5b4-b6b9-40f4-b35e-dae5d3693f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:42.911784 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3c5b4-b6b9-40f4-b35e-dae5d3693f2a None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:42.912064 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3c5b4-b6b9-40f4-b35e-dae5d3693f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:42.912338 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3c5b4-b6b9-40f4-b35e-dae5d3693f2a None admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:42.913372 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3c5b4-b6b9-40f4-b35e-dae5d3693f2a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:42.913680 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3c5b4-b6b9-40f4-b35e-dae5d3693f2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:42.915131 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3c5b4-b6b9-40f4-b35e-dae5d3693f2a None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:42.925368 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:42.933528 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 635/1268] 10.222.0.27 () {56 vars in 1143 bytes} [Sat Jul 16 14:53:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:42.939740 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf771189-ff5e-483e-a425-c18cec00b776 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:43.042437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf771189-ff5e-483e-a425-c18cec00b776 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:43.078212 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952aa366-417f-40c5-9a7c-cc836d557ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:43.079020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952aa366-417f-40c5-9a7c-cc836d557ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:43.079298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952aa366-417f-40c5-9a7c-cc836d557ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:43.079567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952aa366-417f-40c5-9a7c-cc836d557ff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:43.090509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952aa366-417f-40c5-9a7c-cc836d557ff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:43.092578 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952aa366-417f-40c5-9a7c-cc836d557ff2 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:43.110308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-952aa366-417f-40c5-9a7c-cc836d557ff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:43.110308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:43.110308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:43.110308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:43.110308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:43.110308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:43.110308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:43.110308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:43.110308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:43.110308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:43.110308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:43.110308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:43.110308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:53:43.110308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:53:43.110308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:53:43.111315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:53:43.111315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:43.111315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:43.111315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:43.111315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:43.111315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:43.111315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:43.111315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:43.111315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:43.111315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:43.111315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:43.111315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:43.111315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:43.111315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:43.111315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:43.111315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:43.111315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:43.112299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:53:43.112299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:53:43.112299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:43.112299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:53:43.112299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 634/1269] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:53:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:43.117354 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3c5b4-b6b9-40f4-b35e-dae5d3693f2a None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:43.220939 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3c5b4-b6b9-40f4-b35e-dae5d3693f2a None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:43.257418 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f918f7-2539-4475-a22b-37ad2c445c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:43.258083 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f918f7-2539-4475-a22b-37ad2c445c7b None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:43.258365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f918f7-2539-4475-a22b-37ad2c445c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:43.258638 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f918f7-2539-4475-a22b-37ad2c445c7b None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:43.259396 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f918f7-2539-4475-a22b-37ad2c445c7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:43.259702 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f918f7-2539-4475-a22b-37ad2c445c7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:43.261077 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f918f7-2539-4475-a22b-37ad2c445c7b None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:43.273057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 636/1270] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:43.278784 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952aa366-417f-40c5-9a7c-cc836d557ff2 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:43.311516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:43.380534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952aa366-417f-40c5-9a7c-cc836d557ff2 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:43.415722 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98692973-a6d9-404e-a8dc-8ee51ef0235e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:43.416530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98692973-a6d9-404e-a8dc-8ee51ef0235e None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:43.416871 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98692973-a6d9-404e-a8dc-8ee51ef0235e None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:43.417147 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98692973-a6d9-404e-a8dc-8ee51ef0235e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:43.425640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98692973-a6d9-404e-a8dc-8ee51ef0235e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:43.427120 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98692973-a6d9-404e-a8dc-8ee51ef0235e None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:43.441043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-98692973-a6d9-404e-a8dc-8ee51ef0235e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:53:43.441043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:43.441043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:43.441043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:43.441043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:43.441043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:43.441043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:43.441043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:43.441043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:43.441043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:43.441043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:43.441043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:43.441043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:53:43.441043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:53:43.441043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:53:43.442112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:53:43.442112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:43.442112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:43.442112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:43.442112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:43.442112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:43.442112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:43.442112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:43.442112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:43.442112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:43.442112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:43.442112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:43.442112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:43.442112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:43.442112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:43.442112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:43.442112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:43.443078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:53:43.443078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:53:43.443078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:53:43.443078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:53:43.443078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:53:43.443078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:53:43.443078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:53:43.443078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:53:43.443078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 635/1271] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:53:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:43.448576 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f918f7-2539-4475-a22b-37ad2c445c7b None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:43.509426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:43.551573 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f918f7-2539-4475-a22b-37ad2c445c7b None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:43.584384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0f9eb-af9b-4948-828d-ee8b16bae088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:43.585301 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0f9eb-af9b-4948-828d-ee8b16bae088 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:43.585586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0f9eb-af9b-4948-828d-ee8b16bae088 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:43.585861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0f9eb-af9b-4948-828d-ee8b16bae088 None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:43.586652 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0f9eb-af9b-4948-828d-ee8b16bae088 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:43.587001 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0f9eb-af9b-4948-828d-ee8b16bae088 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:43.588440 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0f9eb-af9b-4948-828d-ee8b16bae088 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:43.601742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 637/1272] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:53:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:43.608262 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98692973-a6d9-404e-a8dc-8ee51ef0235e None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:43.706829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:43.715655 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98692973-a6d9-404e-a8dc-8ee51ef0235e None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:43.750063 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ee4c4-6b84-4f80-969a-b8e9c361f0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:43.750977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ee4c4-6b84-4f80-969a-b8e9c361f0bd None admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:43.751260 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ee4c4-6b84-4f80-969a-b8e9c361f0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:43.751542 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ee4c4-6b84-4f80-969a-b8e9c361f0bd None admin] PATH_INFO: `/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/3a30cee766814c8685578819261705ba/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:43.753541 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ee4c4-6b84-4f80-969a-b8e9c361f0bd None admin] RBAC: Authorizing `identity:create_grant(project_id=2739f00eb01e46a9acf90ffa360d7d8c, user_id=3a30cee766814c8685578819261705ba, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:43.755115 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ee4c4-6b84-4f80-969a-b8e9c361f0bd None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:43.871099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 636/1273] 10.222.0.27 () {58 vars in 1439 bytes} [Sat Jul 16 14:53:43 2022] PUT /identity/v3/projects/2739f00eb01e46a9acf90ffa360d7d8c/users/3a30cee766814c8685578819261705ba/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 14:53:45.929871 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ce8ab-b4c0-4b53-b6b3-127cd37828d3 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:45.930394 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ce8ab-b4c0-4b53-b6b3-127cd37828d3 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:45.930516 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ce8ab-b4c0-4b53-b6b3-127cd37828d3 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:45.931990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 638/1274] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:53:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:53:45.937498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2540b-d436-4301-9398-bff948aab220 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:45.937935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2540b-d436-4301-9398-bff948aab220 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:45.938248 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2540b-d436-4301-9398-bff948aab220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:45.955750 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-57a2540b-d436-4301-9398-bff948aab220 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:53:45.956972 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-57a2540b-d436-4301-9398-bff948aab220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:53:46.032220 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 637/1275] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:53:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:53:46.056054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ce8ab-b4c0-4b53-b6b3-127cd37828d3 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:46.169038 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ce8ab-b4c0-4b53-b6b3-127cd37828d3 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:46.189113 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:46.207224 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a803f7b-a89d-4209-9b53-ad05aeb887f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:46.208040 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a803f7b-a89d-4209-9b53-ad05aeb887f9 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:46.208321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a803f7b-a89d-4209-9b53-ad05aeb887f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:46.208590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a803f7b-a89d-4209-9b53-ad05aeb887f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:46.220585 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a803f7b-a89d-4209-9b53-ad05aeb887f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 16 14:53:46.228445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-2a803f7b-a89d-4209-9b53-ad05aeb887f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:53:46.228445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:46.228445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:46.228445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:46.228445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:46.228445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:46.228445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:46.228445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:46.228445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:46.228445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:46.228445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:46.228445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:46.228445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 16 14:53:46.228445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 16 14:53:46.228445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 16 14:53:46.228445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 16 14:53:46.229698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 16 14:53:46.229698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 16 14:53:46.229698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 16 14:53:46.229698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 16 14:53:46.229698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:46.229698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:46.229698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:46.229698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:46.229698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:46.229698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:46.229698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:46.229698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:46.229698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:46.229698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:46.229698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:46.229698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:46.230739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:46.230739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:46.230739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:46.230739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:46.230739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 16 14:53:46.230739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 16 14:53:46.230739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 16 14:53:46.230739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 16 14:53:46.230739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 16 14:53:46.230739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 16 14:53:46.230739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 16 14:53:46.230739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:46.230739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 639/1276] 10.222.0.27 () {56 vars in 1096 bytes} [Sat Jul 16 14:53:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:46.236829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2540b-d436-4301-9398-bff948aab220 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:46.322772 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:46.343172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2540b-d436-4301-9398-bff948aab220 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:46.378698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33301954-c7e8-4079-b034-e41e6ae115a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:46.379431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33301954-c7e8-4079-b034-e41e6ae115a7 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:46.379711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33301954-c7e8-4079-b034-e41e6ae115a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:46.379983 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33301954-c7e8-4079-b034-e41e6ae115a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:46.380730 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33301954-c7e8-4079-b034-e41e6ae115a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:46.381038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33301954-c7e8-4079-b034-e41e6ae115a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:46.382401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33301954-c7e8-4079-b034-e41e6ae115a7 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:46.392441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 638/1277] 10.222.0.27 () {56 vars in 1099 bytes} [Sat Jul 16 14:53:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:46.398419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a803f7b-a89d-4209-9b53-ad05aeb887f9 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:46.501101 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a803f7b-a89d-4209-9b53-ad05aeb887f9 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:46.535416 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e9709-aa95-4e8a-8db0-704a612b0833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:46.536201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e9709-aa95-4e8a-8db0-704a612b0833 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:46.536478 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e9709-aa95-4e8a-8db0-704a612b0833 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:46.536746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e9709-aa95-4e8a-8db0-704a612b0833 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:46.547660 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e9709-aa95-4e8a-8db0-704a612b0833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:46.549206 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e9709-aa95-4e8a-8db0-704a612b0833 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:46.566118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-c33e9709-aa95-4e8a-8db0-704a612b0833 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:46.566118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:46.566118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:46.566118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:46.566118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:46.566118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:46.566118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:46.566118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:46.566118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:46.566118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:46.566118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:46.566118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:46.566118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:53:46.566118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:53:46.566118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:53:46.567248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:53:46.567248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:46.567248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:46.567248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:46.567248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:46.567248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:46.567248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:46.567248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:46.567248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:46.567248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:46.567248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:46.567248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:46.567248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:46.567248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:46.567248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:46.567248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:46.567248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:46.568385 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:53:46.568385 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:53:46.568385 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:46.568385 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:46.568385 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 640/1278] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:53:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:46.573261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33301954-c7e8-4079-b034-e41e6ae115a7 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:46.676570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33301954-c7e8-4079-b034-e41e6ae115a7 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:46.711172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0e135-47c6-4165-8d45-e4893e6239d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:46.711833 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0e135-47c6-4165-8d45-e4893e6239d9 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:46.712115 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0e135-47c6-4165-8d45-e4893e6239d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:46.712386 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0e135-47c6-4165-8d45-e4893e6239d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:46.713115 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb0e135-47c6-4165-8d45-e4893e6239d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:46.713407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb0e135-47c6-4165-8d45-e4893e6239d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:46.714764 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb0e135-47c6-4165-8d45-e4893e6239d9 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:46.727176 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 639/1279] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:46.733395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e9709-aa95-4e8a-8db0-704a612b0833 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:46.765158 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:46.833766 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e9709-aa95-4e8a-8db0-704a612b0833 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:46.869376 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895ad1e-926d-45e6-9bbf-05b8b59e7bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:46.870173 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895ad1e-926d-45e6-9bbf-05b8b59e7bbf None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:46.870442 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895ad1e-926d-45e6-9bbf-05b8b59e7bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:46.870707 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895ad1e-926d-45e6-9bbf-05b8b59e7bbf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:46.878156 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895ad1e-926d-45e6-9bbf-05b8b59e7bbf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:46.879647 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895ad1e-926d-45e6-9bbf-05b8b59e7bbf None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:46.895055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-e895ad1e-926d-45e6-9bbf-05b8b59e7bbf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 16 14:53:46.895055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:46.895055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:46.895055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:46.895055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:46.895055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:46.895055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:46.895055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:46.895055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:46.895055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:46.895055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:46.895055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:46.895055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 16 14:53:46.895055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 16 14:53:46.895055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 16 14:53:46.895055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 16 14:53:46.898956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:46.898956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:46.898956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 16 14:53:46.898956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:53:46.898956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 16 14:53:46.898956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 16 14:53:46.898956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:46.898956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:46.898956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:46.898956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:46.898956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:46.898956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:46.898956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:46.898956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:46.898956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:46.898956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:46.898956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:46.898956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:46.899955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:46.899955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:46.899955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 16 14:53:46.899955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 16 14:53:46.899955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 16 14:53:46.899955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 16 14:53:46.899955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 16 14:53:46.899955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 16 14:53:46.899955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 16 14:53:46.899955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:46.899955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 641/1280] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:53:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:46.902592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0e135-47c6-4165-8d45-e4893e6239d9 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:46.907507 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:47.009206 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0e135-47c6-4165-8d45-e4893e6239d9 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:47.044085 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5ad68-8539-4b85-a511-89e8d7b80273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:47.044677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5ad68-8539-4b85-a511-89e8d7b80273 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:47.044975 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5ad68-8539-4b85-a511-89e8d7b80273 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:47.045798 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5ad68-8539-4b85-a511-89e8d7b80273 None admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:47.046635 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c5ad68-8539-4b85-a511-89e8d7b80273 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:47.046963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c5ad68-8539-4b85-a511-89e8d7b80273 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:47.048404 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c5ad68-8539-4b85-a511-89e8d7b80273 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:47.066217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 640/1281] 10.222.0.27 () {56 vars in 1143 bytes} [Sat Jul 16 14:53:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:47.072207 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895ad1e-926d-45e6-9bbf-05b8b59e7bbf None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:47.166253 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:47.176295 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895ad1e-926d-45e6-9bbf-05b8b59e7bbf None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:47.209782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1c8dd-d3a1-4f54-a9b8-c59cfcbabf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:47.210675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1c8dd-d3a1-4f54-a9b8-c59cfcbabf3f None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:47.210999 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1c8dd-d3a1-4f54-a9b8-c59cfcbabf3f None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:47.211290 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1c8dd-d3a1-4f54-a9b8-c59cfcbabf3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:47.281621 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1c8dd-d3a1-4f54-a9b8-c59cfcbabf3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:47.283089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1c8dd-d3a1-4f54-a9b8-c59cfcbabf3f None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:47.372571 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:47.383474 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 642/1282] 10.222.0.27 () {58 vars in 1282 bytes} [Sat Jul 16 14:53:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 16 14:53:47.390112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5ad68-8539-4b85-a511-89e8d7b80273 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:47.451762 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:47.490276 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5ad68-8539-4b85-a511-89e8d7b80273 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:47.524726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a16c8-d0b1-45ed-bf53-f33615a04e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:47.525510 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a16c8-d0b1-45ed-bf53-f33615a04e8b None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:47.525795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a16c8-d0b1-45ed-bf53-f33615a04e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:47.526069 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a16c8-d0b1-45ed-bf53-f33615a04e8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:47.536661 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a16c8-d0b1-45ed-bf53-f33615a04e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:47.538100 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a16c8-d0b1-45ed-bf53-f33615a04e8b None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:47.555306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-9e0a16c8-d0b1-45ed-bf53-f33615a04e8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:47.555306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:47.555306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:47.555306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:47.555306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:47.555306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:47.555306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:47.555306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:47.555306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:47.555306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:47.555306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:47.555306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:47.555306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 16 14:53:47.555306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 16 14:53:47.555306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 16 14:53:47.556499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 16 14:53:47.556499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:47.556499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:47.556499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:47.556499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:47.556499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:47.556499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:47.556499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:47.556499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:47.556499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:47.556499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:47.556499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:47.556499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:47.556499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:47.556499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:47.556499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:47.556499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:47.557805 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 16 14:53:47.557805 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 16 14:53:47.557805 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 16 14:53:47.557805 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:53:47.557805 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 641/1283] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 14:53:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:47.562387 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1c8dd-d3a1-4f54-a9b8-c59cfcbabf3f None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:47.663594 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1c8dd-d3a1-4f54-a9b8-c59cfcbabf3f None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:47.697099 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde7119-3e17-4c40-9535-fb1f09d45375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:47.697770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde7119-3e17-4c40-9535-fb1f09d45375 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:47.698061 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde7119-3e17-4c40-9535-fb1f09d45375 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:47.698364 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde7119-3e17-4c40-9535-fb1f09d45375 None admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:47.699179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde7119-3e17-4c40-9535-fb1f09d45375 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:47.699486 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde7119-3e17-4c40-9535-fb1f09d45375 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:47.712429 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde7119-3e17-4c40-9535-fb1f09d45375 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:47.713227 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 643/1284] 10.222.0.27 () {56 vars in 1109 bytes} [Sat Jul 16 14:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:53:47.719889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a16c8-d0b1-45ed-bf53-f33615a04e8b None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:47.824215 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a16c8-d0b1-45ed-bf53-f33615a04e8b None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:47.838562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:47.859899 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef77ed-0eff-48c1-bf2a-9de750bd1093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:47.860685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef77ed-0eff-48c1-bf2a-9de750bd1093 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:47.860963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef77ed-0eff-48c1-bf2a-9de750bd1093 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:47.861229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef77ed-0eff-48c1-bf2a-9de750bd1093 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:47.867950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef77ed-0eff-48c1-bf2a-9de750bd1093 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:47.869518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef77ed-0eff-48c1-bf2a-9de750bd1093 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:47.884698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-b2ef77ed-0eff-48c1-bf2a-9de750bd1093 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:53:47.884698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:47.884698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:47.884698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:47.884698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:47.884698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:47.884698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:47.884698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:47.884698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:47.884698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:47.884698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:47.884698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:47.884698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 16 14:53:47.884698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 16 14:53:47.884698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 16 14:53:47.885799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 16 14:53:47.885799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:47.885799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:47.885799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:47.885799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:47.885799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:47.885799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:47.885799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:47.885799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:47.885799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:47.885799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:47.885799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:47.885799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:47.885799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:47.885799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:47.885799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:47.885799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:47.886773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 14:53:47.886773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 14:53:47.886773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 14:53:47.886773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 14:53:47.886773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 14:53:47.886773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 14:53:47.886773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 16 14:53:47.886773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 14:53:47.886773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 642/1285] 10.222.0.27 () {56 vars in 1111 bytes} [Sat Jul 16 14:53:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:47.892221 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde7119-3e17-4c40-9535-fb1f09d45375 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:47.944178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:47.995453 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde7119-3e17-4c40-9535-fb1f09d45375 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:48.027577 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da9097-7f86-408d-a14e-7ef7e8390270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:48.028238 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da9097-7f86-408d-a14e-7ef7e8390270 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:48.028524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da9097-7f86-408d-a14e-7ef7e8390270 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:48.028795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da9097-7f86-408d-a14e-7ef7e8390270 None admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:48.030104 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da9097-7f86-408d-a14e-7ef7e8390270 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:48.030407 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da9097-7f86-408d-a14e-7ef7e8390270 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:48.031888 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da9097-7f86-408d-a14e-7ef7e8390270 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:48.044110 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 644/1286] 10.222.0.27 () {56 vars in 1148 bytes} [Sat Jul 16 14:53:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:48.051306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef77ed-0eff-48c1-bf2a-9de750bd1093 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:48.152611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef77ed-0eff-48c1-bf2a-9de750bd1093 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:48.189629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e443f71-782e-45a0-9532-09ea4a297f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:48.190283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e443f71-782e-45a0-9532-09ea4a297f12 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:48.190558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e443f71-782e-45a0-9532-09ea4a297f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:48.190828 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e443f71-782e-45a0-9532-09ea4a297f12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:48.191691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e443f71-782e-45a0-9532-09ea4a297f12 None admin] RBAC: Adding query filter params (role.id=0efeff46573247f3b9f1e2dec2968672, scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c, user.id=3a30cee766814c8685578819261705ba) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:48.194036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e443f71-782e-45a0-9532-09ea4a297f12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:48.194036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e443f71-782e-45a0-9532-09ea4a297f12 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:48.199689 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 643/1287] 10.222.0.27 () {56 vars in 1374 bytes} [Sat Jul 16 14:53:48 2022] GET /identity/v3/role_assignments?user.id=3a30cee766814c8685578819261705ba&scope.project.id=2739f00eb01e46a9acf90ffa360d7d8c&role.id=0efeff46573247f3b9f1e2dec2968672 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:50.303569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973486e9-3519-4f25-b809-7f95b3aee000 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:50.304091 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973486e9-3519-4f25-b809-7f95b3aee000 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:50.304091 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973486e9-3519-4f25-b809-7f95b3aee000 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:50.305403 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 645/1288] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:53:50.310709 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2e3d2-4b3b-4366-bae3-2f8483a4e2ab None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:50.311203 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2e3d2-4b3b-4366-bae3-2f8483a4e2ab None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:50.311430 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2e3d2-4b3b-4366-bae3-2f8483a4e2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:50.328219 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a4e2e3d2-4b3b-4366-bae3-2f8483a4e2ab None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:53:50.329399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a4e2e3d2-4b3b-4366-bae3-2f8483a4e2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:53:50.360422 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:50.397895 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 644/1289] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:53:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:53:50.422007 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973486e9-3519-4f25-b809-7f95b3aee000 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:50.528475 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973486e9-3519-4f25-b809-7f95b3aee000 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:50.540194 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:50.563960 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21c738-74b3-44c7-b20f-6d57413a2c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:50.564696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21c738-74b3-44c7-b20f-6d57413a2c21 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:50.564980 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21c738-74b3-44c7-b20f-6d57413a2c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:50.565252 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21c738-74b3-44c7-b20f-6d57413a2c21 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:50.565902 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21c738-74b3-44c7-b20f-6d57413a2c21 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:50.567303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21c738-74b3-44c7-b20f-6d57413a2c21 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:50.579642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-1d21c738-74b3-44c7-b20f-6d57413a2c21 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 16 14:53:50.579642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:50.579642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:53:50.579642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:53:50.579642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:53:50.579642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:53:50.579642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:53:50.579642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:53:50.579642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 16 14:53:50.579642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:50.579642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:53:50.579642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:50.579642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:50.579642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:50.579642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:50.579642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:50.579642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:50.580944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:50.580944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:50.580944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:50.580944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:50.580944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:50.580944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:50.580944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:53:50.580944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:53:50.580944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:53:50.580944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:53:50.580944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:50.580944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:50.580944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:50.580944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:50.580944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:50.580944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:50.581825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:50.581825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:50.581825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:50.581825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:50.581825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:50.581825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:50.581825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:50.581825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:50.581825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:50.581825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:50.581825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:53:50.581825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:53:50.581825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 16 14:53:50.581825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:50.581825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 646/1290] 10.222.0.27 () {56 vars in 1117 bytes} [Sat Jul 16 14:53:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 16 14:53:50.587185 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2e3d2-4b3b-4366-bae3-2f8483a4e2ab None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:50.688658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2e3d2-4b3b-4366-bae3-2f8483a4e2ab None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:50.724504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd80be7-45dc-45d7-bc7d-5b9ed0857053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:50.725173 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd80be7-45dc-45d7-bc7d-5b9ed0857053 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:50.725446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd80be7-45dc-45d7-bc7d-5b9ed0857053 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:50.725737 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd80be7-45dc-45d7-bc7d-5b9ed0857053 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:50.726423 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd80be7-45dc-45d7-bc7d-5b9ed0857053 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:50.726724 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd80be7-45dc-45d7-bc7d-5b9ed0857053 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:50.728121 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd80be7-45dc-45d7-bc7d-5b9ed0857053 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:50.733360 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 645/1291] 10.222.0.27 () {56 vars in 1116 bytes} [Sat Jul 16 14:53:50 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:50.739277 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21c738-74b3-44c7-b20f-6d57413a2c21 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:50.840992 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21c738-74b3-44c7-b20f-6d57413a2c21 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:50.875711 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c3c12-068f-4357-a12e-eb097c2f0713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:50.876369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c3c12-068f-4357-a12e-eb097c2f0713 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:50.876648 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c3c12-068f-4357-a12e-eb097c2f0713 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:50.876920 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c3c12-068f-4357-a12e-eb097c2f0713 None admin] PATH_INFO: `/v3/services` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:50.877658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c3c12-068f-4357-a12e-eb097c2f0713 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:50.877959 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c3c12-068f-4357-a12e-eb097c2f0713 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:50.879409 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c3c12-068f-4357-a12e-eb097c2f0713 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:50.884339 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 647/1292] 10.222.0.27 () {56 vars in 1116 bytes} [Sat Jul 16 14:53:50 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:52.985699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271a472-8fbf-4de4-991e-1213e64e77c6 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:52.986263 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271a472-8fbf-4de4-991e-1213e64e77c6 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:52.986263 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271a472-8fbf-4de4-991e-1213e64e77c6 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:52.987657 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 646/1293] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:53:52.992938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03881e2-9c73-4463-a65c-a77d2015cf8c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:52.993375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03881e2-9c73-4463-a65c-a77d2015cf8c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:52.993464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03881e2-9c73-4463-a65c-a77d2015cf8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:53.011398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-e03881e2-9c73-4463-a65c-a77d2015cf8c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:53:53.012577 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e03881e2-9c73-4463-a65c-a77d2015cf8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:53:53.043263 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:53.083831 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 648/1294] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:53:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:53:53.108402 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271a472-8fbf-4de4-991e-1213e64e77c6 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:53.139781 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:53.210000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271a472-8fbf-4de4-991e-1213e64e77c6 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:53.246324 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27566b-5c36-4205-bd72-a7d8f4bc7d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:53.247059 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27566b-5c36-4205-bd72-a7d8f4bc7d1c None admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:53.247345 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27566b-5c36-4205-bd72-a7d8f4bc7d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:53.247632 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27566b-5c36-4205-bd72-a7d8f4bc7d1c None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:53.248871 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27566b-5c36-4205-bd72-a7d8f4bc7d1c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:53.250246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27566b-5c36-4205-bd72-a7d8f4bc7d1c None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:53.374395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 647/1295] 10.222.0.27 () {60 vars in 1141 bytes} [Sat Jul 16 14:53:53 2022] POST /identity/v3/services => generated 246 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:53:55.510849 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc244d-fd4e-4b19-bb45-262202cacb2c None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:55.511446 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc244d-fd4e-4b19-bb45-262202cacb2c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:55.511446 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc244d-fd4e-4b19-bb45-262202cacb2c None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:55.512797 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 649/1296] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:53:55.518811 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c291fc2-1168-4285-858c-98b54a2f84dd None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:55.519308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c291fc2-1168-4285-858c-98b54a2f84dd None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:55.519455 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c291fc2-1168-4285-858c-98b54a2f84dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:55.537242 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-2c291fc2-1168-4285-858c-98b54a2f84dd None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:53:55.538427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2c291fc2-1168-4285-858c-98b54a2f84dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:53:55.613677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 648/1297] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:53:55 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:53:55.636796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc244d-fd4e-4b19-bb45-262202cacb2c None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:55.741461 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc244d-fd4e-4b19-bb45-262202cacb2c None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:55.775670 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca3ff3-645e-4fc4-b4ae-d97dca5d0094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:55.776384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca3ff3-645e-4fc4-b4ae-d97dca5d0094 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:55.776661 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca3ff3-645e-4fc4-b4ae-d97dca5d0094 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:55.776929 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca3ff3-645e-4fc4-b4ae-d97dca5d0094 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:55.777568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca3ff3-645e-4fc4-b4ae-d97dca5d0094 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:55.779667 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca3ff3-645e-4fc4-b4ae-d97dca5d0094 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:55.791496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-b4ca3ff3-645e-4fc4-b4ae-d97dca5d0094 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 16 14:53:55.791496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:55.791496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:53:55.791496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:53:55.791496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:53:55.791496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:53:55.791496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:53:55.791496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:53:55.791496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 16 14:53:55.791496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:55.791496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:53:55.791496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:55.791496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:55.791496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:55.791496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:55.791496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:55.791496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:55.792873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:55.792873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:55.792873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:55.792873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:55.792873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:55.792873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:55.792873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:53:55.792873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:53:55.792873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:53:55.792873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:53:55.792873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:55.792873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:55.792873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:55.792873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:55.792873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:55.792873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:55.793881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:55.793881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:55.793881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:55.793881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:55.793881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:55.793881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:55.793881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:55.793881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:55.793881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:55.793881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:55.793881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:53:55.793881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:53:55.793881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 16 14:53:55.793881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:55.793881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 650/1298] 10.222.0.27 () {56 vars in 1117 bytes} [Sat Jul 16 14:53:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:55.800015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c291fc2-1168-4285-858c-98b54a2f84dd None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:55.819403 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:55.901774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c291fc2-1168-4285-858c-98b54a2f84dd None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:55.936984 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f318e-8563-4bce-8c18-e6c89e9b78c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:55.937645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f318e-8563-4bce-8c18-e6c89e9b78c6 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:55.937920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f318e-8563-4bce-8c18-e6c89e9b78c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:55.938187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f318e-8563-4bce-8c18-e6c89e9b78c6 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:55.938909 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f318e-8563-4bce-8c18-e6c89e9b78c6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:55.939213 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f318e-8563-4bce-8c18-e6c89e9b78c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:55.940570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f318e-8563-4bce-8c18-e6c89e9b78c6 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:55.945573 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 649/1299] 10.222.0.27 () {56 vars in 1116 bytes} [Sat Jul 16 14:53:55 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 14:53:55.951735 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca3ff3-645e-4fc4-b4ae-d97dca5d0094 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:55.985649 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:56.053711 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca3ff3-645e-4fc4-b4ae-d97dca5d0094 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:56.090044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58868bdb-2158-49cf-bdd9-bd679a9c1b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:56.090758 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58868bdb-2158-49cf-bdd9-bd679a9c1b59 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:56.091070 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58868bdb-2158-49cf-bdd9-bd679a9c1b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:56.091864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58868bdb-2158-49cf-bdd9-bd679a9c1b59 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:56.094418 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58868bdb-2158-49cf-bdd9-bd679a9c1b59 None admin] RBAC: Adding query filter params (interface=public, service_id=f7ed3151ff924c7c91d2d02e7834c5d0, region_id=RegionOne) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:56.094418 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58868bdb-2158-49cf-bdd9-bd679a9c1b59 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:56.094418 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58868bdb-2158-49cf-bdd9-bd679a9c1b59 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:56.099882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 651/1300] 10.222.0.27 () {56 vars in 1251 bytes} [Sat Jul 16 14:53:55 2022] GET /identity/v3/endpoints?service_id=f7ed3151ff924c7c91d2d02e7834c5d0&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:56.105583 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f318e-8563-4bce-8c18-e6c89e9b78c6 None admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:56.190381 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:56.208887 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f318e-8563-4bce-8c18-e6c89e9b78c6 None admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:56.244228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3c428-1f9a-4826-b434-0ef8b0660762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:56.244896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3c428-1f9a-4826-b434-0ef8b0660762 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:56.245176 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3c428-1f9a-4826-b434-0ef8b0660762 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:56.245447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3c428-1f9a-4826-b434-0ef8b0660762 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:56.246120 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3c428-1f9a-4826-b434-0ef8b0660762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:56.247546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3c428-1f9a-4826-b434-0ef8b0660762 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:56.254439 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 650/1301] 10.222.0.27 () {56 vars in 1087 bytes} [Sat Jul 16 14:53:56 2022] GET /identity/v3/services => generated 2191 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 14:53:58.320338 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533dc7d-5f70-42bb-9d39-6608effa008d None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:58.320841 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533dc7d-5f70-42bb-9d39-6608effa008d None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:58.320841 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c533dc7d-5f70-42bb-9d39-6608effa008d None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:58.322172 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 652/1302] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 14:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 14:53:58.328172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878cfcc-799e-4855-b851-7d0debf633e6 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:58.337461 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878cfcc-799e-4855-b851-7d0debf633e6 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:58.337461 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878cfcc-799e-4855-b851-7d0debf633e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:58.346090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-7878cfcc-799e-4855-b851-7d0debf633e6 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 14:53:58.347327 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7878cfcc-799e-4855-b851-7d0debf633e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 14:53:58.415201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 651/1303] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 14:53:58 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 16 14:53:58.437725 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533dc7d-5f70-42bb-9d39-6608effa008d None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:58.532569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:58.539030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c533dc7d-5f70-42bb-9d39-6608effa008d None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:58.572135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8fad1-2fa8-45ac-933d-c9a289217663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:58.572849 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8fad1-2fa8-45ac-933d-c9a289217663 None admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:58.573129 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8fad1-2fa8-45ac-933d-c9a289217663 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:58.573397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8fad1-2fa8-45ac-933d-c9a289217663 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:58.574035 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8fad1-2fa8-45ac-933d-c9a289217663 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:58.575421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8fad1-2fa8-45ac-933d-c9a289217663 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:58.588004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application [None req-f3e8fad1-2fa8-45ac-933d-c9a289217663 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 16 14:53:58.588004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:58.588004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 16 14:53:58.588004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 16 14:53:58.588004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 16 14:53:58.588004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 16 14:53:58.588004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 16 14:53:58.588004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:53:58.588004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 16 14:53:58.588004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:58.588004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 16 14:53:58.588004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:58.588004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 14:53:58.588004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 14:53:58.588004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 14:53:58.588004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 14:53:58.588004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 14:53:58.589169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 14:53:58.589169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 14:53:58.589169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 14:53:58.589169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 14:53:58.589169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 14:53:58.589169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 14:53:58.589169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 16 14:53:58.589169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 16 14:53:58.589169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 16 14:53:58.589169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 16 14:53:58.589169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 14:53:58.589169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 14:53:58.589169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 14:53:58.589169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 14:53:58.589169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 14:53:58.589169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 14:53:58.590170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 14:53:58.590170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application with Lock( Jul 16 14:53:58.590170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 14:53:58.590170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self._enter() Jul 16 14:53:58.590170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 14:53:58.590170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 14:53:58.590170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 14:53:58.590170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application return self.creator() Jul 16 14:53:58.590170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 14:53:58.590170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application created_value = creator( Jul 16 14:53:58.590170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 16 14:53:58.590170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 16 14:53:58.590170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 16 14:53:58.590170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: ERROR keystone.server.flask.application Jul 16 14:53:58.590170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 653/1304] 10.222.0.27 () {56 vars in 1117 bytes} [Sat Jul 16 14:53:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 16 14:53:58.595365 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878cfcc-799e-4855-b851-7d0debf633e6 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:58.671645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:53:58.694110 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878cfcc-799e-4855-b851-7d0debf633e6 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:58.726293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c37f8-a5b4-4c01-a2fb-f80d9a45bd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:58.726972 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c37f8-a5b4-4c01-a2fb-f80d9a45bd14 None admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:58.727255 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c37f8-a5b4-4c01-a2fb-f80d9a45bd14 None admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:58.727528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c37f8-a5b4-4c01-a2fb-f80d9a45bd14 None admin] PATH_INFO: `/v3/services` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:58.728223 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c37f8-a5b4-4c01-a2fb-f80d9a45bd14 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 14:53:58.728527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c37f8-a5b4-4c01-a2fb-f80d9a45bd14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:58.729933 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c37f8-a5b4-4c01-a2fb-f80d9a45bd14 None admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:58.735482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 652/1305] 10.222.0.27 () {56 vars in 1116 bytes} [Sat Jul 16 14:53:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 14:53:58.741372 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8fad1-2fa8-45ac-933d-c9a289217663 None admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 14:53:58.841864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8fad1-2fa8-45ac-933d-c9a289217663 None admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 14:53:58.874875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb4d01-d02d-4f98-b323-da3a978d5e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 14:53:58.875568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb4d01-d02d-4f98-b323-da3a978d5e88 None admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 14:53:58.875842 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb4d01-d02d-4f98-b323-da3a978d5e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 14:53:58.876174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb4d01-d02d-4f98-b323-da3a978d5e88 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 14:53:58.876925 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb4d01-d02d-4f98-b323-da3a978d5e88 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 14:53:58.878363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb4d01-d02d-4f98-b323-da3a978d5e88 None admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 14:53:58.978659 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 654/1306] 10.222.0.27 () {60 vars in 1144 bytes} [Sat Jul 16 14:53:58 2022] POST /identity/v3/endpoints => generated 331 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 14:54:08.817884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=30 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 14:54:08.973677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:28.398628 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353711c-dac5-48a8-96c4-47de782bde5b None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:28.399264 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353711c-dac5-48a8-96c4-47de782bde5b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:28.399677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353711c-dac5-48a8-96c4-47de782bde5b None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:28.402302 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 653/1307] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:01:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:01:28.411911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe85e6-6904-479e-bb2a-e670885e333c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:28.412504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe85e6-6904-479e-bb2a-e670885e333c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:28.412833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe85e6-6904-479e-bb2a-e670885e333c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:28.534710 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-dfbe85e6-6904-479e-bb2a-e670885e333c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:01:28.536338 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-dfbe85e6-6904-479e-bb2a-e670885e333c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:28.679443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 655/1308] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:01:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:01:28.733400 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33f139-bed9-4add-8103-c4dccdb391cc None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:28.733688 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33f139-bed9-4add-8103-c4dccdb391cc None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:28.733964 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33f139-bed9-4add-8103-c4dccdb391cc None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:28.735383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 654/1309] 10.222.0.27 () {54 vars in 964 bytes} [Sat Jul 16 15:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 15:01:28.743946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fe52b-9851-45f7-b772-5ae4feea0a02 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:28.744336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fe52b-9851-45f7-b772-5ae4feea0a02 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:28.744794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fe52b-9851-45f7-b772-5ae4feea0a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:28.842057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-5e5fe52b-9851-45f7-b772-5ae4feea0a02 None None] MFA Rules not processed for user `6e372cc62af24664b01a967435eb8902`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:01:28.843259 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5e5fe52b-9851-45f7-b772-5ae4feea0a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:28.868408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:28.959685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33f139-bed9-4add-8103-c4dccdb391cc None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:01:28.961567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1c33f139-bed9-4add-8103-c4dccdb391cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:28.970363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 656/1310] 10.222.0.27 () {58 vars in 1061 bytes} [Sat Jul 16 15:01:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:01:29.099161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33f139-bed9-4add-8103-c4dccdb391cc None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:01:29.200851 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f193f-fa09-42f7-911d-79d089f480ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:01:29.202396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f193f-fa09-42f7-911d-79d089f480ef service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:29.203170 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f193f-fa09-42f7-911d-79d089f480ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:29.203905 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f193f-fa09-42f7-911d-79d089f480ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:29.208423 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-dc2f193f-fa09-42f7-911d-79d089f480ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:29.317085 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:29.410809 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f193f-fa09-42f7-911d-79d089f480ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:01:29.412384 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f193f-fa09-42f7-911d-79d089f480ef service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:01:29.432967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 655/1311] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:01:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:01:31.809030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c8ac6-f262-44b2-bdc6-5d1a10eac0ca None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:31.809556 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c8ac6-f262-44b2-bdc6-5d1a10eac0ca None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:31.809556 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c8ac6-f262-44b2-bdc6-5d1a10eac0ca None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:31.810857 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 657/1312] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:01:31.816802 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b471c9-5290-4752-a917-22294f2fc0f4 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:31.817281 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b471c9-5290-4752-a917-22294f2fc0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:31.817375 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b471c9-5290-4752-a917-22294f2fc0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:31.895597 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-98b471c9-5290-4752-a917-22294f2fc0f4 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:01:31.896805 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-98b471c9-5290-4752-a917-22294f2fc0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:32.000169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 656/1313] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:01:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:01:32.039146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c8ac6-f262-44b2-bdc6-5d1a10eac0ca None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:01:32.040770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-8f1c8ac6-f262-44b2-bdc6-5d1a10eac0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:32.153109 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c8ac6-f262-44b2-bdc6-5d1a10eac0ca None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:01:32.228239 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:32.231156 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ccef81-b85f-4012-ac05-fba3d1266f0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:01:32.231916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccef81-b85f-4012-ac05-fba3d1266f0f service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:32.232106 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccef81-b85f-4012-ac05-fba3d1266f0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:32.232417 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccef81-b85f-4012-ac05-fba3d1266f0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:32.234095 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-64ccef81-b85f-4012-ac05-fba3d1266f0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:32.365001 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ccef81-b85f-4012-ac05-fba3d1266f0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:01:32.366503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ccef81-b85f-4012-ac05-fba3d1266f0f service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:01:32.388841 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 658/1314] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:01:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:01:35.018073 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b66a9c-ea0b-4a22-921a-4a55873429bc None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:35.018605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b66a9c-ea0b-4a22-921a-4a55873429bc None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:35.019076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b66a9c-ea0b-4a22-921a-4a55873429bc None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:35.020461 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 657/1315] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:01:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:01:35.025818 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f2cc4-e820-417f-9ccc-b8e852819c1f None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:35.026221 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f2cc4-e820-417f-9ccc-b8e852819c1f None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:35.026375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f2cc4-e820-417f-9ccc-b8e852819c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:35.092186 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-009f2cc4-e820-417f-9ccc-b8e852819c1f None None] MFA Rules not processed for user `bcbfd29852d047909a1de71fb64876b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:01:35.093400 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-009f2cc4-e820-417f-9ccc-b8e852819c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:35.148245 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:35.196183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 659/1316] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:01:35 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:01:35.236289 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b66a9c-ea0b-4a22-921a-4a55873429bc None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:01:35.238809 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-74b66a9c-ea0b-4a22-921a-4a55873429bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:35.248780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:35.344449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b66a9c-ea0b-4a22-921a-4a55873429bc None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:01:35.419620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c079996-1a84-4088-9e95-2d407908e601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:01:35.420268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c079996-1a84-4088-9e95-2d407908e601 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:35.420549 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c079996-1a84-4088-9e95-2d407908e601 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:35.420824 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c079996-1a84-4088-9e95-2d407908e601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:35.422466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2c079996-1a84-4088-9e95-2d407908e601 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:35.487297 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:35.537275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c079996-1a84-4088-9e95-2d407908e601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:01:35.538707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c079996-1a84-4088-9e95-2d407908e601 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:01:35.561151 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 658/1317] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:01:35 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:01:38.489074 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613293b5-1bb8-4c97-92e6-4640f42830c3 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:38.489634 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613293b5-1bb8-4c97-92e6-4640f42830c3 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:38.489634 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613293b5-1bb8-4c97-92e6-4640f42830c3 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:38.491105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 660/1318] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:01:38.496886 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c2f05-450a-42e0-92f1-779075604741 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:38.497477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c2f05-450a-42e0-92f1-779075604741 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:38.497477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c2f05-450a-42e0-92f1-779075604741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:38.565224 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ff5c2f05-450a-42e0-92f1-779075604741 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:01:38.566444 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ff5c2f05-450a-42e0-92f1-779075604741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:38.666180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 659/1319] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:01:38 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:01:38.706463 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613293b5-1bb8-4c97-92e6-4640f42830c3 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:01:38.708028 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-613293b5-1bb8-4c97-92e6-4640f42830c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:38.813369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613293b5-1bb8-4c97-92e6-4640f42830c3 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:01:38.889414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9ab61-530c-452e-9b86-a0137bcd03b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:01:38.890065 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9ab61-530c-452e-9b86-a0137bcd03b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:38.890349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9ab61-530c-452e-9b86-a0137bcd03b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:38.890620 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9ab61-530c-452e-9b86-a0137bcd03b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:38.892307 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-35f9ab61-530c-452e-9b86-a0137bcd03b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:38.900899 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:39.006783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9ab61-530c-452e-9b86-a0137bcd03b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:01:39.008265 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9ab61-530c-452e-9b86-a0137bcd03b2 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:01:39.027750 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 661/1320] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:01:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:01:42.207205 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71246163-b1dc-4613-8042-80929e22402f None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:42.208003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71246163-b1dc-4613-8042-80929e22402f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:42.208003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71246163-b1dc-4613-8042-80929e22402f None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:42.209272 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 660/1321] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:01:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:01:42.215723 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5699184-4ce2-4d73-9117-9cf1fb93c416 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:42.216218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5699184-4ce2-4d73-9117-9cf1fb93c416 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:42.216739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5699184-4ce2-4d73-9117-9cf1fb93c416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:42.293196 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-e5699184-4ce2-4d73-9117-9cf1fb93c416 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:01:42.294520 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e5699184-4ce2-4d73-9117-9cf1fb93c416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:42.359020 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:42.408388 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 662/1322] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:01:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:01:42.446459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71246163-b1dc-4613-8042-80929e22402f None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:01:42.448081 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-71246163-b1dc-4613-8042-80929e22402f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:42.468848 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:42.571892 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71246163-b1dc-4613-8042-80929e22402f None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:01:42.667134 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0431c18-b24a-40fc-b8e0-e82a2a6c33ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:01:42.667804 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0431c18-b24a-40fc-b8e0-e82a2a6c33ec service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:42.668085 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0431c18-b24a-40fc-b8e0-e82a2a6c33ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:42.668356 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0431c18-b24a-40fc-b8e0-e82a2a6c33ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:42.670135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d0431c18-b24a-40fc-b8e0-e82a2a6c33ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:42.762225 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:42.814715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0431c18-b24a-40fc-b8e0-e82a2a6c33ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:01:42.816456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0431c18-b24a-40fc-b8e0-e82a2a6c33ec service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:01:42.836482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 661/1323] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:01:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:01:45.008290 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2254b2d-61d7-4073-b787-ad0c315adeaf None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:45.008996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2254b2d-61d7-4073-b787-ad0c315adeaf None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:45.008996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2254b2d-61d7-4073-b787-ad0c315adeaf None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:45.010402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 663/1324] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:01:45.015829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deede377-e3b7-4528-a2bf-506af18d03e8 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:45.016317 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deede377-e3b7-4528-a2bf-506af18d03e8 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:45.016409 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deede377-e3b7-4528-a2bf-506af18d03e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:45.092392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-deede377-e3b7-4528-a2bf-506af18d03e8 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:01:45.093518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-deede377-e3b7-4528-a2bf-506af18d03e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:45.208360 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 662/1325] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:01:45.262235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49a402-0353-4653-b66c-fd813d80efee None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:45.262514 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49a402-0353-4653-b66c-fd813d80efee None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:45.262995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49a402-0353-4653-b66c-fd813d80efee None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:45.264264 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 664/1326] 10.222.0.27 () {54 vars in 964 bytes} [Sat Jul 16 15:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:01:45.272162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0adb98-79a3-470f-a1ac-d126e271ba01 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:45.272162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0adb98-79a3-470f-a1ac-d126e271ba01 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:45.272423 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0adb98-79a3-470f-a1ac-d126e271ba01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:45.306926 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:45.360842 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-0c0adb98-79a3-470f-a1ac-d126e271ba01 None None] MFA Rules not processed for user `6e372cc62af24664b01a967435eb8902`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:01:45.362862 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-0c0adb98-79a3-470f-a1ac-d126e271ba01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:45.459992 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 663/1327] 10.222.0.27 () {58 vars in 1061 bytes} [Sat Jul 16 15:01:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:01:45.470146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49a402-0353-4653-b66c-fd813d80efee None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:01:45.471967 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5e49a402-0353-4653-b66c-fd813d80efee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:45.598385 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49a402-0353-4653-b66c-fd813d80efee None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:01:45.686359 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b37738-e523-4ce8-b525-60bd1e001ad5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:01:45.686954 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b37738-e523-4ce8-b525-60bd1e001ad5 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:45.687237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b37738-e523-4ce8-b525-60bd1e001ad5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:45.687508 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b37738-e523-4ce8-b525-60bd1e001ad5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:45.689356 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b3b37738-e523-4ce8-b525-60bd1e001ad5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:45.721226 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:45.824358 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b37738-e523-4ce8-b525-60bd1e001ad5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:01:45.826070 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b37738-e523-4ce8-b525-60bd1e001ad5 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:01:45.846179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 665/1328] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:01:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:01:48.200737 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43006372-705a-4cb1-8b93-ac6a65a64b8e None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:48.201251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43006372-705a-4cb1-8b93-ac6a65a64b8e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:48.201251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43006372-705a-4cb1-8b93-ac6a65a64b8e None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:48.202577 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 664/1329] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:01:48.209025 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d259b-547d-41de-b78f-7377c77d301c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:48.209553 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d259b-547d-41de-b78f-7377c77d301c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:48.209553 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d259b-547d-41de-b78f-7377c77d301c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:48.277732 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-856d259b-547d-41de-b78f-7377c77d301c None None] MFA Rules not processed for user `bcbfd29852d047909a1de71fb64876b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:01:48.278975 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-856d259b-547d-41de-b78f-7377c77d301c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:48.345301 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:48.383910 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 666/1330] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:01:48 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:01:48.422393 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43006372-705a-4cb1-8b93-ac6a65a64b8e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:01:48.424702 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-43006372-705a-4cb1-8b93-ac6a65a64b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:48.555238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43006372-705a-4cb1-8b93-ac6a65a64b8e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:01:48.577056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:48.634089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744fec8b-d06f-4ea5-980d-8fa6f7260e4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:01:48.634777 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fec8b-d06f-4ea5-980d-8fa6f7260e4b service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:48.635082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fec8b-d06f-4ea5-980d-8fa6f7260e4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:48.635354 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744fec8b-d06f-4ea5-980d-8fa6f7260e4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:48.637088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-744fec8b-d06f-4ea5-980d-8fa6f7260e4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:48.756549 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744fec8b-d06f-4ea5-980d-8fa6f7260e4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:01:48.757909 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744fec8b-d06f-4ea5-980d-8fa6f7260e4b service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:01:48.778123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 665/1331] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:01:48 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:01:51.750727 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630ca15-cd6a-41fc-bd77-623e51bebb51 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:51.751330 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630ca15-cd6a-41fc-bd77-623e51bebb51 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:51.751330 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630ca15-cd6a-41fc-bd77-623e51bebb51 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:51.752778 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 667/1332] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:01:51.758673 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89131741-7dd1-4e4c-b56b-360a77c90f81 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:51.759212 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89131741-7dd1-4e4c-b56b-360a77c90f81 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:51.759319 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89131741-7dd1-4e4c-b56b-360a77c90f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:51.824976 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:51.828040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-89131741-7dd1-4e4c-b56b-360a77c90f81 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:01:51.829287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-89131741-7dd1-4e4c-b56b-360a77c90f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:51.927725 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 666/1333] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:01:51.965013 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630ca15-cd6a-41fc-bd77-623e51bebb51 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:01:51.966661 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e630ca15-cd6a-41fc-bd77-623e51bebb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:52.075202 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630ca15-cd6a-41fc-bd77-623e51bebb51 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:01:52.152554 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68eb91-82f3-410f-ba7f-d35e42473a8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:01:52.153205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68eb91-82f3-410f-ba7f-d35e42473a8e service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:52.153485 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68eb91-82f3-410f-ba7f-d35e42473a8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:52.153756 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68eb91-82f3-410f-ba7f-d35e42473a8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:52.155512 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-9b68eb91-82f3-410f-ba7f-d35e42473a8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:52.184797 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:52.276543 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68eb91-82f3-410f-ba7f-d35e42473a8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:01:52.278030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68eb91-82f3-410f-ba7f-d35e42473a8e service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:01:52.300886 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 668/1334] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:01:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:01:54.404854 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5736c2-59b7-49ea-92cd-c270190cdf59 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:54.405421 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5736c2-59b7-49ea-92cd-c270190cdf59 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:54.405573 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5736c2-59b7-49ea-92cd-c270190cdf59 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:54.407014 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 667/1335] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:01:54.412751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431afe9-c225-4034-9b60-1cddab37f3b3 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:54.413198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431afe9-c225-4034-9b60-1cddab37f3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:54.413329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431afe9-c225-4034-9b60-1cddab37f3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:54.479869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-5431afe9-c225-4034-9b60-1cddab37f3b3 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:01:54.481063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5431afe9-c225-4034-9b60-1cddab37f3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:54.558747 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:54.582267 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 669/1336] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:01:54.623198 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5736c2-59b7-49ea-92cd-c270190cdf59 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:01:54.624720 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4e5736c2-59b7-49ea-92cd-c270190cdf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:54.731557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5736c2-59b7-49ea-92cd-c270190cdf59 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:01:54.761820 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:54.816208 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4550d-cd48-4069-b73d-5a6a03c6f2d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:01:54.816862 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4550d-cd48-4069-b73d-5a6a03c6f2d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:54.817143 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4550d-cd48-4069-b73d-5a6a03c6f2d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:54.817415 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4550d-cd48-4069-b73d-5a6a03c6f2d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:54.819106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-aab4550d-cd48-4069-b73d-5a6a03c6f2d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:54.932951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4550d-cd48-4069-b73d-5a6a03c6f2d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:01:54.934392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab4550d-cd48-4069-b73d-5a6a03c6f2d7 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:01:54.954177 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 668/1337] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:01:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:01:57.737794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48e7c8-b3f9-4a7f-8258-dc4c4871dd55 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:57.738615 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48e7c8-b3f9-4a7f-8258-dc4c4871dd55 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:57.738779 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48e7c8-b3f9-4a7f-8258-dc4c4871dd55 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:57.740212 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 670/1338] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:01:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:01:57.745715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ee702-5948-47dd-a9ca-2cae633d371d None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:57.746202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ee702-5948-47dd-a9ca-2cae633d371d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:57.746299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ee702-5948-47dd-a9ca-2cae633d371d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:57.818635 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-4f2ee702-5948-47dd-a9ca-2cae633d371d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:01:57.819836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4f2ee702-5948-47dd-a9ca-2cae633d371d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:57.829164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:57.924117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 669/1339] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:01:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:01:57.961019 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48e7c8-b3f9-4a7f-8258-dc4c4871dd55 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:01:57.962808 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ee48e7c8-b3f9-4a7f-8258-dc4c4871dd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:58.069169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48e7c8-b3f9-4a7f-8258-dc4c4871dd55 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:01:58.148099 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284664e-cdd5-42ef-b1fd-40110c0a05ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:01:58.148748 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284664e-cdd5-42ef-b1fd-40110c0a05ef service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:01:58.149029 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284664e-cdd5-42ef-b1fd-40110c0a05ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:01:58.149301 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284664e-cdd5-42ef-b1fd-40110c0a05ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:01:58.151025 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b284664e-cdd5-42ef-b1fd-40110c0a05ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:01:58.194249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:01:58.273861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b284664e-cdd5-42ef-b1fd-40110c0a05ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:01:58.275350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b284664e-cdd5-42ef-b1fd-40110c0a05ef service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:01:58.294725 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 671/1340] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:01:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:02:02.870963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865445f-6803-41c5-b74c-15b2a8b03f29 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:02.871501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865445f-6803-41c5-b74c-15b2a8b03f29 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:02.871501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865445f-6803-41c5-b74c-15b2a8b03f29 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:02.874455 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 670/1341] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:02:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:02.881224 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10f826-c4ed-44fa-a476-d0f12cb2866c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:02.881224 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10f826-c4ed-44fa-a476-d0f12cb2866c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:02.881224 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10f826-c4ed-44fa-a476-d0f12cb2866c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:02.964199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-bf10f826-c4ed-44fa-a476-d0f12cb2866c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:02.965605 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-bf10f826-c4ed-44fa-a476-d0f12cb2866c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:03.070962 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:03.085551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 672/1342] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:02:03.125125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865445f-6803-41c5-b74c-15b2a8b03f29 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:02:03.127428 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5865445f-6803-41c5-b74c-15b2a8b03f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:03.264894 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865445f-6803-41c5-b74c-15b2a8b03f29 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:02:03.312768 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:03.363796 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb405c-ecda-4b55-93be-a38e6b5ad92b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:02:03.364625 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb405c-ecda-4b55-93be-a38e6b5ad92b service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:03.364911 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb405c-ecda-4b55-93be-a38e6b5ad92b service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:03.365189 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb405c-ecda-4b55-93be-a38e6b5ad92b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:03.366975 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a0eb405c-ecda-4b55-93be-a38e6b5ad92b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:03.571491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb405c-ecda-4b55-93be-a38e6b5ad92b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:02:03.574199 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb405c-ecda-4b55-93be-a38e6b5ad92b service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:02:03.610523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 671/1343] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:02:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:02:06.875138 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05e0b5-2127-454a-ae23-74f114b3ebad None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:06.875731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05e0b5-2127-454a-ae23-74f114b3ebad None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:06.875731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05e0b5-2127-454a-ae23-74f114b3ebad None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:06.877597 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 673/1344] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:02:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:06.883298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb600f85-9508-42fe-b284-6587643815a7 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:06.883802 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb600f85-9508-42fe-b284-6587643815a7 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:06.883916 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb600f85-9508-42fe-b284-6587643815a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:06.961665 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-eb600f85-9508-42fe-b284-6587643815a7 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:06.962793 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-eb600f85-9508-42fe-b284-6587643815a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:06.982931 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:07.070079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 672/1345] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:02:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:02:07.109624 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05e0b5-2127-454a-ae23-74f114b3ebad None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:02:07.112201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-0e05e0b5-2127-454a-ae23-74f114b3ebad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:07.225483 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05e0b5-2127-454a-ae23-74f114b3ebad None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:02:07.307452 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e55557-ac08-48d5-b9ce-e84f2b4ee225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:02:07.308110 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e55557-ac08-48d5-b9ce-e84f2b4ee225 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:07.308386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e55557-ac08-48d5-b9ce-e84f2b4ee225 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:07.308656 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e55557-ac08-48d5-b9ce-e84f2b4ee225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:07.310287 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c6e55557-ac08-48d5-b9ce-e84f2b4ee225 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:07.365206 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:07.435922 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e55557-ac08-48d5-b9ce-e84f2b4ee225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:02:07.437371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e55557-ac08-48d5-b9ce-e84f2b4ee225 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:02:07.459583 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 674/1346] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:02:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:02:11.984131 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8c974-87e5-4fcd-a2bb-94fb9fa0513e None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:11.984658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8c974-87e5-4fcd-a2bb-94fb9fa0513e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:11.984658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8c974-87e5-4fcd-a2bb-94fb9fa0513e None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:11.987001 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 673/1347] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:02:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:11.992699 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58a58f-50ab-49a3-9fea-03720ed467a8 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:11.993232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58a58f-50ab-49a3-9fea-03720ed467a8 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:11.993339 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58a58f-50ab-49a3-9fea-03720ed467a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:12.074053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-7f58a58f-50ab-49a3-9fea-03720ed467a8 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:12.075627 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7f58a58f-50ab-49a3-9fea-03720ed467a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:12.201200 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:12.204594 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 675/1348] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:02:12.245559 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8c974-87e5-4fcd-a2bb-94fb9fa0513e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:02:12.247090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d4e8c974-87e5-4fcd-a2bb-94fb9fa0513e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:12.367560 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8c974-87e5-4fcd-a2bb-94fb9fa0513e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:02:12.430080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:12.470790 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9907bd-5383-4156-8535-2e0e57732bbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:02:12.471520 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9907bd-5383-4156-8535-2e0e57732bbd service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:12.471801 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9907bd-5383-4156-8535-2e0e57732bbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:12.472075 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9907bd-5383-4156-8535-2e0e57732bbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:12.473969 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-cd9907bd-5383-4156-8535-2e0e57732bbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:12.643291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9907bd-5383-4156-8535-2e0e57732bbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:02:12.644631 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9907bd-5383-4156-8535-2e0e57732bbd service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:02:12.663010 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 674/1349] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:02:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:02:18.309493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745bb37-512b-47a7-8411-1c29351c7f53 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:18.310300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745bb37-512b-47a7-8411-1c29351c7f53 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:18.310479 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745bb37-512b-47a7-8411-1c29351c7f53 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:18.311967 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 676/1350] 10.222.0.27 () {54 vars in 921 bytes} [Sat Jul 16 15:02:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:18.319084 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb71de-c408-4ff2-bdcf-2df1780b79b2 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:18.319819 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb71de-c408-4ff2-bdcf-2df1780b79b2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:18.319819 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb71de-c408-4ff2-bdcf-2df1780b79b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:18.419489 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c9eb71de-c408-4ff2-bdcf-2df1780b79b2 None None] MFA Rules not processed for user `3a30cee766814c8685578819261705ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:18.424353 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c9eb71de-c408-4ff2-bdcf-2df1780b79b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:18.448067 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:18.636802 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 675/1351] 10.222.0.27 () {58 vars in 1018 bytes} [Sat Jul 16 15:02:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:02:23.539209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06e31d-9d66-4ff6-a408-0f6a5845d59c None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:23.539209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06e31d-9d66-4ff6-a408-0f6a5845d59c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:23.540055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06e31d-9d66-4ff6-a408-0f6a5845d59c None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:23.540896 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 677/1352] 10.222.0.27 () {54 vars in 921 bytes} [Sat Jul 16 15:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:23.547000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e1f07-d8c9-4d89-bad0-e18dcac4fae1 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:23.547575 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e1f07-d8c9-4d89-bad0-e18dcac4fae1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:23.548421 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e1f07-d8c9-4d89-bad0-e18dcac4fae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:23.654287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-826e1f07-d8c9-4d89-bad0-e18dcac4fae1 None None] MFA Rules not processed for user `3a30cee766814c8685578819261705ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:23.655563 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-826e1f07-d8c9-4d89-bad0-e18dcac4fae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:23.759884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:23.808010 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 676/1353] 10.222.0.27 () {58 vars in 1018 bytes} [Sat Jul 16 15:02:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:02:25.163958 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcda1fe-c4f1-42b5-94b9-e459beaf67ab None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:25.164495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcda1fe-c4f1-42b5-94b9-e459beaf67ab None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:25.164495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcda1fe-c4f1-42b5-94b9-e459beaf67ab None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:25.165916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 678/1354] 10.222.0.27 () {54 vars in 921 bytes} [Sat Jul 16 15:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:25.171582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affc9de-6923-4f38-b59e-c29de0e59507 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:25.172083 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affc9de-6923-4f38-b59e-c29de0e59507 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:25.177097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affc9de-6923-4f38-b59e-c29de0e59507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:25.255796 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-2affc9de-6923-4f38-b59e-c29de0e59507 None None] MFA Rules not processed for user `3a30cee766814c8685578819261705ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:25.257019 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2affc9de-6923-4f38-b59e-c29de0e59507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:25.374745 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 677/1355] 10.222.0.27 () {58 vars in 1018 bytes} [Sat Jul 16 15:02:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:02:29.149494 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5fbf8-08f9-482d-baf4-8f6b3875bf77 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:29.150978 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5fbf8-08f9-482d-baf4-8f6b3875bf77 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:29.151627 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5fbf8-08f9-482d-baf4-8f6b3875bf77 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:29.153446 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 679/1356] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:02:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:29.160210 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d84ec-ec83-4aa3-9af0-c2cb86c97d08 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:29.160868 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d84ec-ec83-4aa3-9af0-c2cb86c97d08 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:29.160868 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d84ec-ec83-4aa3-9af0-c2cb86c97d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:29.202375 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:29.270344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f58d84ec-ec83-4aa3-9af0-c2cb86c97d08 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:29.274561 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f58d84ec-ec83-4aa3-9af0-c2cb86c97d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:29.478700 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 678/1357] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:02:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:02:29.596979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c82619-f34e-4f9e-a646-6d8390952d5d None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:29.597670 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c82619-f34e-4f9e-a646-6d8390952d5d None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:29.598278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c82619-f34e-4f9e-a646-6d8390952d5d None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:29.602265 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 680/1358] 10.222.0.27 () {54 vars in 961 bytes} [Sat Jul 16 15:02:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:29.615792 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acdfff-832f-404f-8436-fd9e85bb2d59 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:29.617167 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acdfff-832f-404f-8436-fd9e85bb2d59 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:29.617167 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acdfff-832f-404f-8436-fd9e85bb2d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:29.742282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a8acdfff-832f-404f-8436-fd9e85bb2d59 None None] MFA Rules not processed for user `3f2e241494244edf8d283e8f44192c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:29.744103 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a8acdfff-832f-404f-8436-fd9e85bb2d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:29.821143 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:29.909973 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 679/1359] 10.222.0.27 () {58 vars in 1058 bytes} [Sat Jul 16 15:02:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:02:29.918041 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c82619-f34e-4f9e-a646-6d8390952d5d None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:02:29.919616 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-09c82619-f34e-4f9e-a646-6d8390952d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:30.095180 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c82619-f34e-4f9e-a646-6d8390952d5d None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:02:30.196643 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:30.245669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16c4f4-31d5-464f-83e4-c31505c26a99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:02:30.246399 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16c4f4-31d5-464f-83e4-c31505c26a99 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:30.246686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16c4f4-31d5-464f-83e4-c31505c26a99 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:30.246993 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16c4f4-31d5-464f-83e4-c31505c26a99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:30.248801 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-8f16c4f4-31d5-464f-83e4-c31505c26a99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:30.374961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16c4f4-31d5-464f-83e4-c31505c26a99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:02:30.376865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16c4f4-31d5-464f-83e4-c31505c26a99 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:02:30.400957 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 681/1360] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:02:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:02:33.022580 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c1510-b69a-4b70-aede-2a72ea347e28 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:33.023203 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c1510-b69a-4b70-aede-2a72ea347e28 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:33.023203 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c1510-b69a-4b70-aede-2a72ea347e28 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:33.025994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 680/1361] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:02:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:33.030226 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ceb0a6-4cd1-4107-b6f2-ae238a39a628 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:33.030734 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ceb0a6-4cd1-4107-b6f2-ae238a39a628 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:33.030853 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ceb0a6-4cd1-4107-b6f2-ae238a39a628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:33.118763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-64ceb0a6-4cd1-4107-b6f2-ae238a39a628 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:33.121113 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-64ceb0a6-4cd1-4107-b6f2-ae238a39a628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:33.181119 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:33.309529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 682/1362] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:02:33.367732 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c1510-b69a-4b70-aede-2a72ea347e28 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:02:33.369252 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-335c1510-b69a-4b70-aede-2a72ea347e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:33.560526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c1510-b69a-4b70-aede-2a72ea347e28 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:02:33.571444 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:33.708975 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d137015-1bae-4482-8794-9c0f15ee3bf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:02:33.709761 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d137015-1bae-4482-8794-9c0f15ee3bf0 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:33.710107 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d137015-1bae-4482-8794-9c0f15ee3bf0 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:33.710446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d137015-1bae-4482-8794-9c0f15ee3bf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:33.712273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2d137015-1bae-4482-8794-9c0f15ee3bf0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:33.832142 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d137015-1bae-4482-8794-9c0f15ee3bf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:02:33.834554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d137015-1bae-4482-8794-9c0f15ee3bf0 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:02:33.868145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 681/1363] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:02:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:02:36.529506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d438db-6f7e-4749-844b-0f0c2fdc5222 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:36.530094 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d438db-6f7e-4749-844b-0f0c2fdc5222 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:36.530374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d438db-6f7e-4749-844b-0f0c2fdc5222 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:36.532543 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 683/1364] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:02:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 15:02:36.539350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75ac42-733f-49f9-81f3-2b351a8dd990 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:36.539983 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75ac42-733f-49f9-81f3-2b351a8dd990 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:36.540258 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75ac42-733f-49f9-81f3-2b351a8dd990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:36.562289 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:36.625451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3f75ac42-733f-49f9-81f3-2b351a8dd990 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:36.627000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3f75ac42-733f-49f9-81f3-2b351a8dd990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:36.791007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 682/1365] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:02:36.861909 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af008f-36f4-4a8f-bb55-8e6134cba98e None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:36.862355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af008f-36f4-4a8f-bb55-8e6134cba98e None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:36.862616 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af008f-36f4-4a8f-bb55-8e6134cba98e None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:36.864821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 684/1366] 10.222.0.27 () {54 vars in 961 bytes} [Sat Jul 16 15:02:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:36.873607 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df827c3-a9bd-42fc-85f7-7a87428345e9 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:36.874324 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df827c3-a9bd-42fc-85f7-7a87428345e9 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:36.874685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df827c3-a9bd-42fc-85f7-7a87428345e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:36.991033 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3df827c3-a9bd-42fc-85f7-7a87428345e9 None None] MFA Rules not processed for user `3f2e241494244edf8d283e8f44192c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:36.992284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3df827c3-a9bd-42fc-85f7-7a87428345e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:37.035272 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:37.145875 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 683/1367] 10.222.0.27 () {58 vars in 1058 bytes} [Sat Jul 16 15:02:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:02:37.159363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af008f-36f4-4a8f-bb55-8e6134cba98e None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:02:37.162401 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a1af008f-36f4-4a8f-bb55-8e6134cba98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:37.356937 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af008f-36f4-4a8f-bb55-8e6134cba98e None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:02:37.380803 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:37.497961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77206cf-6c73-436d-9809-3d70b0dd0c82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:02:37.499169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77206cf-6c73-436d-9809-3d70b0dd0c82 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:37.499453 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77206cf-6c73-436d-9809-3d70b0dd0c82 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:37.499910 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77206cf-6c73-436d-9809-3d70b0dd0c82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:37.502540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b77206cf-6c73-436d-9809-3d70b0dd0c82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:37.616761 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77206cf-6c73-436d-9809-3d70b0dd0c82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:02:37.618482 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77206cf-6c73-436d-9809-3d70b0dd0c82 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:02:37.638614 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 685/1368] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:02:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:02:38.468155 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90279c2-94d4-4164-9318-d2184c97e4ae None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:38.468985 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90279c2-94d4-4164-9318-d2184c97e4ae None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:38.468985 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90279c2-94d4-4164-9318-d2184c97e4ae None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:38.470935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 684/1369] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:38.478279 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f0bcf-d878-498b-b265-6d4341b78790 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:38.478757 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f0bcf-d878-498b-b265-6d4341b78790 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:38.479266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f0bcf-d878-498b-b265-6d4341b78790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:38.525693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:38.571237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-e64f0bcf-d878-498b-b265-6d4341b78790 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:38.572482 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e64f0bcf-d878-498b-b265-6d4341b78790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:38.693887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 686/1370] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:02:39.017069 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22c9d3-d579-4811-baae-f56f52e2f3ba None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:39.017523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22c9d3-d579-4811-baae-f56f52e2f3ba None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:39.017974 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22c9d3-d579-4811-baae-f56f52e2f3ba None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:39.020680 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 685/1371] 10.222.0.27 () {54 vars in 963 bytes} [Sat Jul 16 15:02:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:39.029738 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51369320-a5fd-450f-b7a8-7cbb32177945 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:39.030412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51369320-a5fd-450f-b7a8-7cbb32177945 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:39.030725 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51369320-a5fd-450f-b7a8-7cbb32177945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:39.132587 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-51369320-a5fd-450f-b7a8-7cbb32177945 None None] MFA Rules not processed for user `8c21d12c977e4c36b4f0ed442b11ef49`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:39.134720 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-51369320-a5fd-450f-b7a8-7cbb32177945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:39.207602 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:39.250221 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 687/1372] 10.222.0.27 () {58 vars in 1060 bytes} [Sat Jul 16 15:02:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:02:39.259320 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22c9d3-d579-4811-baae-f56f52e2f3ba None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:02:39.261994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7e22c9d3-d579-4811-baae-f56f52e2f3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:39.409332 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22c9d3-d579-4811-baae-f56f52e2f3ba None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:02:39.477125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:39.522792 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f92ce-20b5-4fb1-8890-e1f6ea86547e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:02:39.523909 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f92ce-20b5-4fb1-8890-e1f6ea86547e service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:39.524476 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f92ce-20b5-4fb1-8890-e1f6ea86547e service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:39.524994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f92ce-20b5-4fb1-8890-e1f6ea86547e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:39.527116 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2b7f92ce-20b5-4fb1-8890-e1f6ea86547e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:39.676061 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f92ce-20b5-4fb1-8890-e1f6ea86547e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:02:39.677518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f92ce-20b5-4fb1-8890-e1f6ea86547e service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:02:39.709366 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 686/1373] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:02:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:02:39.955571 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafca3e-f2c5-494b-aa06-7fee1f6fc893 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:39.955571 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafca3e-f2c5-494b-aa06-7fee1f6fc893 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:39.955571 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafca3e-f2c5-494b-aa06-7fee1f6fc893 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:39.962277 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 688/1374] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:02:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:39.963518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b5eef-1190-423a-ba43-18025671dc37 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:39.963518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b5eef-1190-423a-ba43-18025671dc37 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:39.963518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b5eef-1190-423a-ba43-18025671dc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:40.055528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-344b5eef-1190-423a-ba43-18025671dc37 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:40.057567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-344b5eef-1190-423a-ba43-18025671dc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:40.097645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:40.198350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 687/1375] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:02:40.260504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafca3e-f2c5-494b-aa06-7fee1f6fc893 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:02:40.262042 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7eafca3e-f2c5-494b-aa06-7fee1f6fc893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:40.432235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafca3e-f2c5-494b-aa06-7fee1f6fc893 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:02:40.490077 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:40.567618 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198f106d-5e0f-46ff-b973-8b548cdeadb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:02:40.568281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f106d-5e0f-46ff-b973-8b548cdeadb6 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:40.568562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f106d-5e0f-46ff-b973-8b548cdeadb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:40.568836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198f106d-5e0f-46ff-b973-8b548cdeadb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:40.570628 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-198f106d-5e0f-46ff-b973-8b548cdeadb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:40.708236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f106d-5e0f-46ff-b973-8b548cdeadb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:02:40.710037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198f106d-5e0f-46ff-b973-8b548cdeadb6 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:02:40.730350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 689/1376] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:02:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:02:43.030322 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509cccc-21a1-4705-a11c-49d4bfa51c0e None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:43.030952 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509cccc-21a1-4705-a11c-49d4bfa51c0e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:43.030952 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509cccc-21a1-4705-a11c-49d4bfa51c0e None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:43.032336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 688/1377] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:43.038712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c070dec-203b-441a-a891-8afab195e35b None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:43.039541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c070dec-203b-441a-a891-8afab195e35b None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:43.039704 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c070dec-203b-441a-a891-8afab195e35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:43.115034 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-2c070dec-203b-441a-a891-8afab195e35b None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:43.116241 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-2c070dec-203b-441a-a891-8afab195e35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:43.123415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:43.257453 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 690/1378] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:02:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:02:43.305196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509cccc-21a1-4705-a11c-49d4bfa51c0e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:02:43.307487 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c509cccc-21a1-4705-a11c-49d4bfa51c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:43.531111 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509cccc-21a1-4705-a11c-49d4bfa51c0e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:02:43.536283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:43.714408 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46e425-07e3-4734-9442-a48a98716af4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:02:43.715582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46e425-07e3-4734-9442-a48a98716af4 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:43.716071 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46e425-07e3-4734-9442-a48a98716af4 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:43.716546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46e425-07e3-4734-9442-a48a98716af4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:43.720868 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3c46e425-07e3-4734-9442-a48a98716af4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:43.885697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46e425-07e3-4734-9442-a48a98716af4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:02:43.887587 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46e425-07e3-4734-9442-a48a98716af4 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:02:43.914686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 689/1379] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:02:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:02:46.383696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969dc35e-928f-48ec-b494-6b297f312242 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:46.384295 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969dc35e-928f-48ec-b494-6b297f312242 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:46.384295 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969dc35e-928f-48ec-b494-6b297f312242 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:46.385959 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 691/1380] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:46.392256 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde36aad-efd5-4c8b-9315-a899cae44f7d None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:46.393084 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde36aad-efd5-4c8b-9315-a899cae44f7d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:46.393084 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde36aad-efd5-4c8b-9315-a899cae44f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:46.404056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:46.474599 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-bde36aad-efd5-4c8b-9315-a899cae44f7d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:46.476164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bde36aad-efd5-4c8b-9315-a899cae44f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:46.633284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 690/1381] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:02:46.700885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969dc35e-928f-48ec-b494-6b297f312242 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:02:46.704382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-969dc35e-928f-48ec-b494-6b297f312242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:46.868937 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:46.884579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969dc35e-928f-48ec-b494-6b297f312242 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:02:47.025987 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52299ea0-e1ea-41fe-8e6e-39280a653069 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:02:47.026606 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52299ea0-e1ea-41fe-8e6e-39280a653069 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:47.026914 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52299ea0-e1ea-41fe-8e6e-39280a653069 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:47.027199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52299ea0-e1ea-41fe-8e6e-39280a653069 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:47.028884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-52299ea0-e1ea-41fe-8e6e-39280a653069 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:47.141431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52299ea0-e1ea-41fe-8e6e-39280a653069 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:02:47.143312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52299ea0-e1ea-41fe-8e6e-39280a653069 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:02:47.157882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:47.168266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 692/1382] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:02:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:02:49.761084 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be099ebb-2ab5-4027-86e7-9ac7364f2a57 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:49.761897 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be099ebb-2ab5-4027-86e7-9ac7364f2a57 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:49.761897 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be099ebb-2ab5-4027-86e7-9ac7364f2a57 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:49.763255 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 691/1383] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:49.770227 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bda00-cb8f-481c-94ef-26678ab096e0 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:49.770772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bda00-cb8f-481c-94ef-26678ab096e0 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:49.770772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bda00-cb8f-481c-94ef-26678ab096e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:49.846444 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-ad2bda00-cb8f-481c-94ef-26678ab096e0 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:49.847617 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ad2bda00-cb8f-481c-94ef-26678ab096e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:49.959215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 693/1384] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:02:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:02:50.020624 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be099ebb-2ab5-4027-86e7-9ac7364f2a57 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:02:50.022143 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-be099ebb-2ab5-4027-86e7-9ac7364f2a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:50.133558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:50.204621 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be099ebb-2ab5-4027-86e7-9ac7364f2a57 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:02:50.342462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39ccdbe-a784-4434-aaa7-b06e33e1543a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:02:50.343478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ccdbe-a784-4434-aaa7-b06e33e1543a service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:50.343930 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ccdbe-a784-4434-aaa7-b06e33e1543a service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:50.344773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39ccdbe-a784-4434-aaa7-b06e33e1543a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:50.347524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c39ccdbe-a784-4434-aaa7-b06e33e1543a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:50.407412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:50.452180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39ccdbe-a784-4434-aaa7-b06e33e1543a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:02:50.455249 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39ccdbe-a784-4434-aaa7-b06e33e1543a service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:02:50.477952 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 692/1385] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:02:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:02:52.961637 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d367e67-4911-49e0-a3b4-051f020f1c4b None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:52.962318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d367e67-4911-49e0-a3b4-051f020f1c4b None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:52.962434 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d367e67-4911-49e0-a3b4-051f020f1c4b None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:52.964226 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 694/1386] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:02:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:52.971411 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da3512-7679-4d93-8257-5fa18b7216c2 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:52.971923 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da3512-7679-4d93-8257-5fa18b7216c2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:52.972032 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da3512-7679-4d93-8257-5fa18b7216c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:53.061641 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-01da3512-7679-4d93-8257-5fa18b7216c2 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:53.063585 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-01da3512-7679-4d93-8257-5fa18b7216c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:53.191916 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 693/1387] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:02:53.249722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d367e67-4911-49e0-a3b4-051f020f1c4b None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:02:53.251541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-0d367e67-4911-49e0-a3b4-051f020f1c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:53.359243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:53.451859 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d367e67-4911-49e0-a3b4-051f020f1c4b None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:02:53.598554 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b848f3-c1d2-49d8-bcfa-c8ca51519c16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:02:53.599190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b848f3-c1d2-49d8-bcfa-c8ca51519c16 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:53.600209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b848f3-c1d2-49d8-bcfa-c8ca51519c16 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:53.600209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b848f3-c1d2-49d8-bcfa-c8ca51519c16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:53.601782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-01b848f3-c1d2-49d8-bcfa-c8ca51519c16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:53.651394 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:53.717544 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b848f3-c1d2-49d8-bcfa-c8ca51519c16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:02:53.720283 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b848f3-c1d2-49d8-bcfa-c8ca51519c16 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:02:53.746980 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 695/1388] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:02:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:02:56.311042 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad1f4d-1e03-4d40-8fe5-c2e94dec081c None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:56.311572 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad1f4d-1e03-4d40-8fe5-c2e94dec081c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:56.311572 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad1f4d-1e03-4d40-8fe5-c2e94dec081c None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:56.313554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 694/1389] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:02:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:56.321285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1d2bd-fcbc-43d0-9c62-e2fe9b1535c8 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:56.321823 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1d2bd-fcbc-43d0-9c62-e2fe9b1535c8 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:56.321823 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1d2bd-fcbc-43d0-9c62-e2fe9b1535c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:56.400204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-66a1d2bd-fcbc-43d0-9c62-e2fe9b1535c8 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:56.401475 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-66a1d2bd-fcbc-43d0-9c62-e2fe9b1535c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:56.522958 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 696/1390] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:02:56.570791 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad1f4d-1e03-4d40-8fe5-c2e94dec081c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:02:56.572472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-8cad1f4d-1e03-4d40-8fe5-c2e94dec081c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:56.607946 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:56.773454 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad1f4d-1e03-4d40-8fe5-c2e94dec081c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:02:56.896185 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:56.917391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345016f-fe84-426b-8a0f-dd263a8b92ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:02:56.918484 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345016f-fe84-426b-8a0f-dd263a8b92ce service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:56.919007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345016f-fe84-426b-8a0f-dd263a8b92ce service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:56.919499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345016f-fe84-426b-8a0f-dd263a8b92ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:56.923335 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c345016f-fe84-426b-8a0f-dd263a8b92ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:57.039461 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345016f-fe84-426b-8a0f-dd263a8b92ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:02:57.041069 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345016f-fe84-426b-8a0f-dd263a8b92ce service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:02:57.061539 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 695/1391] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:02:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:02:58.165054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb981ae-ba62-49b6-a68c-01c161256585 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:58.165637 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb981ae-ba62-49b6-a68c-01c161256585 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:58.166153 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb981ae-ba62-49b6-a68c-01c161256585 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:58.168958 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 697/1392] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:02:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:58.177469 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b361a36-896b-4bcb-96e6-cb24bfc76361 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:58.177469 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b361a36-896b-4bcb-96e6-cb24bfc76361 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:58.177469 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b361a36-896b-4bcb-96e6-cb24bfc76361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:58.263189 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-4b361a36-896b-4bcb-96e6-cb24bfc76361 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:58.265348 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4b361a36-896b-4bcb-96e6-cb24bfc76361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:58.342601 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:58.406526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 696/1393] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:02:58.484246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f8696-3b44-4c5e-9208-22e986856ef3 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:58.485166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f8696-3b44-4c5e-9208-22e986856ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:58.485524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f8696-3b44-4c5e-9208-22e986856ef3 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:58.488187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 698/1394] 10.222.0.27 () {54 vars in 960 bytes} [Sat Jul 16 15:02:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:58.497622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad403b-ffef-42fa-978f-99295ab61d66 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:58.497876 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad403b-ffef-42fa-978f-99295ab61d66 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:58.498197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ad403b-ffef-42fa-978f-99295ab61d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:58.599273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b1ad403b-ffef-42fa-978f-99295ab61d66 None None] MFA Rules not processed for user `7d689162291f4f4b97e886ea20d3a8b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:58.600557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b1ad403b-ffef-42fa-978f-99295ab61d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:58.693863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 697/1395] 10.222.0.27 () {58 vars in 1057 bytes} [Sat Jul 16 15:02:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:02:58.702079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f8696-3b44-4c5e-9208-22e986856ef3 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:02:58.703851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c71f8696-3b44-4c5e-9208-22e986856ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:58.720681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:58.834987 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f8696-3b44-4c5e-9208-22e986856ef3 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:02:58.945265 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb73e7a-c367-4a97-b5dc-80f9b7cb70c9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d689162291f4f4b97e886ea20d3a8b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:02:58.945922 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb73e7a-c367-4a97-b5dc-80f9b7cb70c9 service glance] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:58.946207 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb73e7a-c367-4a97-b5dc-80f9b7cb70c9 service glance] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:58.946487 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb73e7a-c367-4a97-b5dc-80f9b7cb70c9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:58.948451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-1cb73e7a-c367-4a97-b5dc-80f9b7cb70c9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:59.056828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:59.107805 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb73e7a-c367-4a97-b5dc-80f9b7cb70c9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:02:59.109271 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb73e7a-c367-4a97-b5dc-80f9b7cb70c9 service glance] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:02:59.142545 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 699/1396] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:02:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:02:59.338693 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90042df2-921a-4a0b-a43a-ae3126f15034 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:59.339269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90042df2-921a-4a0b-a43a-ae3126f15034 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:59.339670 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90042df2-921a-4a0b-a43a-ae3126f15034 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:59.342249 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 698/1397] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:02:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:02:59.352683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a2cac-c200-4e8c-8f66-f167844807d6 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:02:59.353237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a2cac-c200-4e8c-8f66-f167844807d6 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:02:59.353720 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a2cac-c200-4e8c-8f66-f167844807d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:02:59.443184 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-b70a2cac-c200-4e8c-8f66-f167844807d6 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:02:59.444458 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b70a2cac-c200-4e8c-8f66-f167844807d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:59.564227 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 700/1398] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:02:59.623946 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90042df2-921a-4a0b-a43a-ae3126f15034 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:02:59.625528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-90042df2-921a-4a0b-a43a-ae3126f15034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:02:59.700176 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:02:59.826877 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90042df2-921a-4a0b-a43a-ae3126f15034 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:03:00.015485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1c2e1-0fa6-4fc3-8cab-c58e1db41641 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:03:00.016983 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1c2e1-0fa6-4fc3-8cab-c58e1db41641 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:00.017623 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1c2e1-0fa6-4fc3-8cab-c58e1db41641 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:00.017901 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1c2e1-0fa6-4fc3-8cab-c58e1db41641 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:00.019619 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f1b1c2e1-0fa6-4fc3-8cab-c58e1db41641 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:00.043580 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:00.129755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe8ad1-b388-4337-8347-eb127a77742f None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:00.130202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1c2e1-0fa6-4fc3-8cab-c58e1db41641 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:03:00.130415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe8ad1-b388-4337-8347-eb127a77742f None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:00.130575 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe8ad1-b388-4337-8347-eb127a77742f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:00.131874 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1c2e1-0fa6-4fc3-8cab-c58e1db41641 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:03:00.161395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 699/1399] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:02:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:03:00.166135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:00.219681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-00fe8ad1-b388-4337-8347-eb127a77742f None None] MFA Rules not processed for user `a4f499707998403d8f833eb783599466`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:00.221024 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-00fe8ad1-b388-4337-8347-eb127a77742f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:00.322779 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 701/1400] 10.222.0.27 () {58 vars in 1009 bytes} [Sat Jul 16 15:03:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:03:00.368645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbacb2f-5400-46f0-a70b-4d58366ff299 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:00.369339 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbacb2f-5400-46f0-a70b-4d58366ff299 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:00.369610 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbacb2f-5400-46f0-a70b-4d58366ff299 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:00.372343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 700/1401] 10.222.0.27 () {54 vars in 942 bytes} [Sat Jul 16 15:03:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 15:03:00.380611 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd11c4-3d6f-4527-a61d-82b79ad96066 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:00.381378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd11c4-3d6f-4527-a61d-82b79ad96066 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:00.381378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd11c4-3d6f-4527-a61d-82b79ad96066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:00.479963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-67bd11c4-3d6f-4527-a61d-82b79ad96066 None None] MFA Rules not processed for user `e1464c83fb294a3d90e5763ef648e834`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:00.481735 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-67bd11c4-3d6f-4527-a61d-82b79ad96066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:00.576951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:00.652972 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 702/1402] 10.222.0.27 () {58 vars in 1039 bytes} [Sat Jul 16 15:03:00 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:03:00.668948 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbacb2f-5400-46f0-a70b-4d58366ff299 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:03:00.671104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ffbacb2f-5400-46f0-a70b-4d58366ff299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:00.849177 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbacb2f-5400-46f0-a70b-4d58366ff299 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:03:00.866975 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:00.999038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b825211-d0fa-42aa-bc87-f0b20d2ff391 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1464c83fb294a3d90e5763ef648e834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:03:00.999793 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b825211-d0fa-42aa-bc87-f0b20d2ff391 service swift] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:01.000128 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b825211-d0fa-42aa-bc87-f0b20d2ff391 service swift] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:01.000434 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b825211-d0fa-42aa-bc87-f0b20d2ff391 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:01.003002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7b825211-d0fa-42aa-bc87-f0b20d2ff391 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:01.123529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b825211-d0fa-42aa-bc87-f0b20d2ff391 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:03:01.126660 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b825211-d0fa-42aa-bc87-f0b20d2ff391 service swift] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:03:01.140579 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 701/1403] 10.222.0.27 () {60 vars in 1384 bytes} [Sat Jul 16 15:03:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 15:03:02.695553 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fe0c9-50f2-4b07-ae93-9a15cb39d815 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:02.696549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fe0c9-50f2-4b07-ae93-9a15cb39d815 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:02.696549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139fe0c9-50f2-4b07-ae93-9a15cb39d815 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:02.698847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 703/1404] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:03:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:03:02.704144 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b6320-e3a5-4677-8060-eae7a45083a1 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:02.704743 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b6320-e3a5-4677-8060-eae7a45083a1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:02.705248 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b6320-e3a5-4677-8060-eae7a45083a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:02.790840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:02.811956 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-893b6320-e3a5-4677-8060-eae7a45083a1 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:02.813206 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-893b6320-e3a5-4677-8060-eae7a45083a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:02.963029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 702/1405] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:03:03.015723 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fe0c9-50f2-4b07-ae93-9a15cb39d815 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:03:03.017770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-139fe0c9-50f2-4b07-ae93-9a15cb39d815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:03.220318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139fe0c9-50f2-4b07-ae93-9a15cb39d815 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:03:03.251518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:03.369413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7adc2b7-5854-4a45-bc25-0fdd045a699f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:03:03.370069 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adc2b7-5854-4a45-bc25-0fdd045a699f service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:03.370348 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adc2b7-5854-4a45-bc25-0fdd045a699f service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:03.370621 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adc2b7-5854-4a45-bc25-0fdd045a699f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:03.373257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a7adc2b7-5854-4a45-bc25-0fdd045a699f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:03.531716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7adc2b7-5854-4a45-bc25-0fdd045a699f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:03:03.533674 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7adc2b7-5854-4a45-bc25-0fdd045a699f service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:03:03.567654 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 704/1406] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:03:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:03:06.162617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18b515-bae8-4b81-8cdc-7185bbf2827e None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:06.163336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18b515-bae8-4b81-8cdc-7185bbf2827e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:06.163336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18b515-bae8-4b81-8cdc-7185bbf2827e None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:06.164592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 703/1407] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:03:06.170299 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd4c2c-d539-4b1d-80f4-32fb256c7735 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:06.178414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd4c2c-d539-4b1d-80f4-32fb256c7735 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:06.178414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd4c2c-d539-4b1d-80f4-32fb256c7735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:06.221550 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:06.261292 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-e9dd4c2c-d539-4b1d-80f4-32fb256c7735 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:06.262544 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e9dd4c2c-d539-4b1d-80f4-32fb256c7735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:06.425777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 705/1408] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:03:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:03:06.483848 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18b515-bae8-4b81-8cdc-7185bbf2827e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:03:06.488958 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9d18b515-bae8-4b81-8cdc-7185bbf2827e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:06.668230 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:06.706189 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18b515-bae8-4b81-8cdc-7185bbf2827e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:03:06.867712 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871503d1-395b-4dc3-99fe-8f548ded835a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:03:06.868406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871503d1-395b-4dc3-99fe-8f548ded835a service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:06.868693 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871503d1-395b-4dc3-99fe-8f548ded835a service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:06.868975 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871503d1-395b-4dc3-99fe-8f548ded835a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:06.870908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-871503d1-395b-4dc3-99fe-8f548ded835a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:06.983133 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:06.997295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871503d1-395b-4dc3-99fe-8f548ded835a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:03:06.999784 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871503d1-395b-4dc3-99fe-8f548ded835a service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:03:07.026234 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 704/1409] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:03:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:03:16.420995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=51 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:17.037611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=8 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:25.442414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28667e48-d858-4e00-b365-902beffd8686 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:25.443057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28667e48-d858-4e00-b365-902beffd8686 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:25.443057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28667e48-d858-4e00-b365-902beffd8686 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:25.444505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 706/1410] 10.222.0.27 () {54 vars in 918 bytes} [Sat Jul 16 15:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:03:25.450336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00488ecd-5a9e-4a52-8235-1c3aff272017 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:25.451012 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00488ecd-5a9e-4a52-8235-1c3aff272017 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:25.451012 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00488ecd-5a9e-4a52-8235-1c3aff272017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:25.536731 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-00488ecd-5a9e-4a52-8235-1c3aff272017 None None] MFA Rules not processed for user `3a30cee766814c8685578819261705ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:25.538060 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-00488ecd-5a9e-4a52-8235-1c3aff272017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:25.648531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 705/1411] 10.222.0.27 () {58 vars in 1015 bytes} [Sat Jul 16 15:03:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:03:25.664625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565a6ae-104f-4702-b1d1-33588ee78c89 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:25.664909 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565a6ae-104f-4702-b1d1-33588ee78c89 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:25.665265 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565a6ae-104f-4702-b1d1-33588ee78c89 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:25.666619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 707/1412] 10.222.0.27 () {54 vars in 918 bytes} [Sat Jul 16 15:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:03:25.672816 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569414b1-5c72-41d9-acef-5c87671f161e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:25.673100 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569414b1-5c72-41d9-acef-5c87671f161e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:25.673375 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569414b1-5c72-41d9-acef-5c87671f161e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:25.755395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:25.758870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-569414b1-5c72-41d9-acef-5c87671f161e None None] MFA Rules not processed for user `3f2e241494244edf8d283e8f44192c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:25.760127 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-569414b1-5c72-41d9-acef-5c87671f161e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:25.886000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 706/1413] 10.222.0.27 () {58 vars in 1015 bytes} [Sat Jul 16 15:03:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:03:28.423889 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac50c4-0ac0-44e8-9d87-557ed27a6f8e None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:28.424466 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac50c4-0ac0-44e8-9d87-557ed27a6f8e None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:28.424466 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac50c4-0ac0-44e8-9d87-557ed27a6f8e None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:28.425790 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 15:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:03:28.431271 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a58f08-3ff1-4b6b-ae63-49fb44340fb8 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:28.431720 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a58f08-3ff1-4b6b-ae63-49fb44340fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:28.431818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a58f08-3ff1-4b6b-ae63-49fb44340fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:28.501148 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-e5a58f08-3ff1-4b6b-ae63-49fb44340fb8 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:28.502342 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e5a58f08-3ff1-4b6b-ae63-49fb44340fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:28.562994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:28.606701 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 15:03:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:03:28.650371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac50c4-0ac0-44e8-9d87-557ed27a6f8e None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:03:28.652341 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5dac50c4-0ac0-44e8-9d87-557ed27a6f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:28.806075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac50c4-0ac0-44e8-9d87-557ed27a6f8e None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:03:28.887480 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:28.932695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db050b-c61a-4295-a71d-c2b113c0e1c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:03:28.933353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db050b-c61a-4295-a71d-c2b113c0e1c1 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:28.933639 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db050b-c61a-4295-a71d-c2b113c0e1c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:28.933914 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db050b-c61a-4295-a71d-c2b113c0e1c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:28.935654 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-f7db050b-c61a-4295-a71d-c2b113c0e1c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:29.033120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db050b-c61a-4295-a71d-c2b113c0e1c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:03:29.034805 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db050b-c61a-4295-a71d-c2b113c0e1c1 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:03:29.055987 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 709/1416] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:03:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:03:29.970558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87b0cc-9599-4b2b-9627-6b1ffb00283d None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:29.971194 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87b0cc-9599-4b2b-9627-6b1ffb00283d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:29.971194 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87b0cc-9599-4b2b-9627-6b1ffb00283d None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:29.972594 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 15:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:03:29.978043 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37906fa1-5410-497d-9f58-4be1f406085c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:29.978507 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37906fa1-5410-497d-9f58-4be1f406085c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:29.978615 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37906fa1-5410-497d-9f58-4be1f406085c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:30.058269 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-37906fa1-5410-497d-9f58-4be1f406085c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:30.059496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-37906fa1-5410-497d-9f58-4be1f406085c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:30.089971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:30.166064 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 15:03:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:03:30.318065 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87b0cc-9599-4b2b-9627-6b1ffb00283d None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:03:30.319615 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5b87b0cc-9599-4b2b-9627-6b1ffb00283d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:30.480321 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87b0cc-9599-4b2b-9627-6b1ffb00283d None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:03:30.522573 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:30.600887 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b463fc-b15e-4d2d-807a-5b6b7a5ab3d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:03:30.601534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b463fc-b15e-4d2d-807a-5b6b7a5ab3d9 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:30.601811 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b463fc-b15e-4d2d-807a-5b6b7a5ab3d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:30.602082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b463fc-b15e-4d2d-807a-5b6b7a5ab3d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:30.603763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f5b463fc-b15e-4d2d-807a-5b6b7a5ab3d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:30.699050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b463fc-b15e-4d2d-807a-5b6b7a5ab3d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:03:30.700960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b463fc-b15e-4d2d-807a-5b6b7a5ab3d9 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:03:30.721177 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 709/1419] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:03:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:03:31.616939 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c8840-d95a-4c0b-82f3-fd285328e9c6 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:31.617397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c8840-d95a-4c0b-82f3-fd285328e9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:31.617493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c8840-d95a-4c0b-82f3-fd285328e9c6 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:31.618918 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 15:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:03:31.624103 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8406a7d-fbcb-4fa0-ade5-3a5095c12c02 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:31.624767 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8406a7d-fbcb-4fa0-ade5-3a5095c12c02 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:31.624767 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8406a7d-fbcb-4fa0-ade5-3a5095c12c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:31.679523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:31.700830 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b8406a7d-fbcb-4fa0-ade5-3a5095c12c02 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:31.702062 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b8406a7d-fbcb-4fa0-ade5-3a5095c12c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:31.801527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 15:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:03:31.957534 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c8840-d95a-4c0b-82f3-fd285328e9c6 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:03:31.959573 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e28c8840-d95a-4c0b-82f3-fd285328e9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:32.127821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c8840-d95a-4c0b-82f3-fd285328e9c6 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:03:32.140869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:32.246639 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e3109c-5b4d-476b-a689-b4a30dab13fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:03:32.247229 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3109c-5b4d-476b-a689-b4a30dab13fc service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:32.247510 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3109c-5b4d-476b-a689-b4a30dab13fc service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:32.247782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3109c-5b4d-476b-a689-b4a30dab13fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:32.251027 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-47e3109c-5b4d-476b-a689-b4a30dab13fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:32.347028 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e3109c-5b4d-476b-a689-b4a30dab13fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:03:32.348677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e3109c-5b4d-476b-a689-b4a30dab13fc service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:03:32.368819 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 712/1422] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:03:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:03:33.253792 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c6249-a51d-4f5a-9d31-78d22e4140d2 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:33.254308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c6249-a51d-4f5a-9d31-78d22e4140d2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:33.254512 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c6249-a51d-4f5a-9d31-78d22e4140d2 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:33.256118 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 15:03:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:03:33.261773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abaec0-b815-4e22-b672-f217c4c8dd4b None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:33.262237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abaec0-b815-4e22-b672-f217c4c8dd4b None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:33.262416 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abaec0-b815-4e22-b672-f217c4c8dd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:33.283055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:33.341979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-49abaec0-b815-4e22-b672-f217c4c8dd4b None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:33.344149 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-49abaec0-b815-4e22-b672-f217c4c8dd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:33.443397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 15:03:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:03:33.598057 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c6249-a51d-4f5a-9d31-78d22e4140d2 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:03:33.599518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-193c6249-a51d-4f5a-9d31-78d22e4140d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:33.712440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:33.750846 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193c6249-a51d-4f5a-9d31-78d22e4140d2 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:03:33.868868 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d0a7e5-9a04-4908-8ea7-63509047083f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:03:33.869511 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0a7e5-9a04-4908-8ea7-63509047083f service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:33.869791 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0a7e5-9a04-4908-8ea7-63509047083f service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:33.870055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0a7e5-9a04-4908-8ea7-63509047083f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:33.871715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a8d0a7e5-9a04-4908-8ea7-63509047083f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:33.954697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:33.968820 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d0a7e5-9a04-4908-8ea7-63509047083f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:03:33.970472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d0a7e5-9a04-4908-8ea7-63509047083f service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:03:33.990011 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 712/1425] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:03:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:03:34.867055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85383920-08b0-4088-be78-363f7d5de623 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:34.867655 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85383920-08b0-4088-be78-363f7d5de623 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:34.867655 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85383920-08b0-4088-be78-363f7d5de623 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:34.868996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 15:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 15:03:34.874620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee918e95-8c07-4042-9785-f62dd8884ef6 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:34.875142 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee918e95-8c07-4042-9785-f62dd8884ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:34.875249 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee918e95-8c07-4042-9785-f62dd8884ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:34.954215 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ee918e95-8c07-4042-9785-f62dd8884ef6 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:34.955456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ee918e95-8c07-4042-9785-f62dd8884ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:35.054414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 15:03:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:03:35.213937 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85383920-08b0-4088-be78-363f7d5de623 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:03:35.215589 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-85383920-08b0-4088-be78-363f7d5de623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:35.305764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:35.369622 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85383920-08b0-4088-be78-363f7d5de623 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:03:35.485303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784f83c-2af8-476f-9650-3e7e93b206dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:03:35.485858 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784f83c-2af8-476f-9650-3e7e93b206dc service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:35.486135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784f83c-2af8-476f-9650-3e7e93b206dc service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:35.486403 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784f83c-2af8-476f-9650-3e7e93b206dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:35.488073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3784f83c-2af8-476f-9650-3e7e93b206dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:35.539198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:35.586201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784f83c-2af8-476f-9650-3e7e93b206dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:03:35.587852 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784f83c-2af8-476f-9650-3e7e93b206dc service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:03:35.607940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 715/1428] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:03:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:03:36.473786 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514255f8-c708-4cc6-b1db-c50f1452a5d4 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:36.474376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514255f8-c708-4cc6-b1db-c50f1452a5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:36.474376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514255f8-c708-4cc6-b1db-c50f1452a5d4 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:36.475782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 15:03:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:03:36.481070 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1d1aa-1b1b-4db0-869e-fdd76284d5a9 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:36.481570 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1d1aa-1b1b-4db0-869e-fdd76284d5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:36.481674 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1d1aa-1b1b-4db0-869e-fdd76284d5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:36.559072 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-58b1d1aa-1b1b-4db0-869e-fdd76284d5a9 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:36.560287 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-58b1d1aa-1b1b-4db0-869e-fdd76284d5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:36.658333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 15:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:03:36.809080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514255f8-c708-4cc6-b1db-c50f1452a5d4 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:03:36.810517 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-514255f8-c708-4cc6-b1db-c50f1452a5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:36.862559 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:36.966415 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514255f8-c708-4cc6-b1db-c50f1452a5d4 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:03:37.088009 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143530e-e950-410e-833e-12a02d184bc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:03:37.088589 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143530e-e950-410e-833e-12a02d184bc4 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:37.088852 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143530e-e950-410e-833e-12a02d184bc4 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:37.089126 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143530e-e950-410e-833e-12a02d184bc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:37.090858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e143530e-e950-410e-833e-12a02d184bc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:37.098683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:37.182599 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143530e-e950-410e-833e-12a02d184bc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:03:37.184243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143530e-e950-410e-833e-12a02d184bc4 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:03:37.206179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 715/1431] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:03:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:03:38.089920 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e78a8a-4847-449f-9367-59b989ae37cc None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:38.090547 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e78a8a-4847-449f-9367-59b989ae37cc None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:38.090547 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e78a8a-4847-449f-9367-59b989ae37cc None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:38.091988 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 15:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:03:38.098340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9420f83-5fb5-4f85-88f3-ecac0fe1f835 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:38.098873 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9420f83-5fb5-4f85-88f3-ecac0fe1f835 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:38.098873 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9420f83-5fb5-4f85-88f3-ecac0fe1f835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:38.183990 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d9420f83-5fb5-4f85-88f3-ecac0fe1f835 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:38.183990 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d9420f83-5fb5-4f85-88f3-ecac0fe1f835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:38.247645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:38.276514 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 15:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:03:38.428331 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e78a8a-4847-449f-9367-59b989ae37cc None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:03:38.430021 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-05e78a8a-4847-449f-9367-59b989ae37cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:38.448545 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:38.589000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e78a8a-4847-449f-9367-59b989ae37cc None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:03:38.678954 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:38.706388 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0804c0b4-fa9b-4fae-a581-c93375ed75ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:03:38.707056 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804c0b4-fa9b-4fae-a581-c93375ed75ae service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:38.707343 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804c0b4-fa9b-4fae-a581-c93375ed75ae service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:38.707615 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804c0b4-fa9b-4fae-a581-c93375ed75ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:38.709300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-0804c0b4-fa9b-4fae-a581-c93375ed75ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:38.807297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0804c0b4-fa9b-4fae-a581-c93375ed75ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:03:38.808911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0804c0b4-fa9b-4fae-a581-c93375ed75ae service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:03:38.829477 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 718/1434] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:03:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:03:39.750721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206af1e-1dc9-415e-8643-56566804d696 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:39.751330 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206af1e-1dc9-415e-8643-56566804d696 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:39.751330 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206af1e-1dc9-415e-8643-56566804d696 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:39.752680 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 15:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:03:39.757729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248d77c-1e97-4ffa-8adc-9668fb2597ab None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:39.758164 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248d77c-1e97-4ffa-8adc-9668fb2597ab None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:39.758277 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248d77c-1e97-4ffa-8adc-9668fb2597ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:39.835302 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-3248d77c-1e97-4ffa-8adc-9668fb2597ab None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:39.836558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3248d77c-1e97-4ffa-8adc-9668fb2597ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:39.878260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:39.934095 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 15:03:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:03:40.089913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206af1e-1dc9-415e-8643-56566804d696 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:03:40.091339 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7206af1e-1dc9-415e-8643-56566804d696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:40.239141 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206af1e-1dc9-415e-8643-56566804d696 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:03:40.295853 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:40.359340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46592d34-575a-4dd4-bccd-8c5b0d0c05e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:03:40.359989 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46592d34-575a-4dd4-bccd-8c5b0d0c05e1 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:40.360262 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46592d34-575a-4dd4-bccd-8c5b0d0c05e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:40.360529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46592d34-575a-4dd4-bccd-8c5b0d0c05e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:40.362163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-46592d34-575a-4dd4-bccd-8c5b0d0c05e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:40.452808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46592d34-575a-4dd4-bccd-8c5b0d0c05e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:03:40.454416 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46592d34-575a-4dd4-bccd-8c5b0d0c05e1 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:03:40.472653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 718/1437] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:03:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:03:41.395421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df495224-8a5b-426a-9ba4-ac5e24123f90 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:41.396001 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df495224-8a5b-426a-9ba4-ac5e24123f90 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:41.396001 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df495224-8a5b-426a-9ba4-ac5e24123f90 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:41.397391 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 15:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:03:41.402880 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77914d71-1cca-450a-b0d3-00f2023ff436 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:41.403460 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77914d71-1cca-450a-b0d3-00f2023ff436 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:41.403460 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77914d71-1cca-450a-b0d3-00f2023ff436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:41.484386 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-77914d71-1cca-450a-b0d3-00f2023ff436 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:41.485641 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-77914d71-1cca-450a-b0d3-00f2023ff436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:41.487820 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:41.591001 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 15:03:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:03:41.744011 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df495224-8a5b-426a-9ba4-ac5e24123f90 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:03:41.745434 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-df495224-8a5b-426a-9ba4-ac5e24123f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:42.013862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df495224-8a5b-426a-9ba4-ac5e24123f90 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:03:42.040834 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:42.131659 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65cac7-adfb-46f3-b8a3-f67ec3ed6ea7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:03:42.132301 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65cac7-adfb-46f3-b8a3-f67ec3ed6ea7 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:42.132578 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65cac7-adfb-46f3-b8a3-f67ec3ed6ea7 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:42.132848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65cac7-adfb-46f3-b8a3-f67ec3ed6ea7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:42.134471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-dc65cac7-adfb-46f3-b8a3-f67ec3ed6ea7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:42.225582 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65cac7-adfb-46f3-b8a3-f67ec3ed6ea7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:03:42.227218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65cac7-adfb-46f3-b8a3-f67ec3ed6ea7 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:03:42.245673 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 721/1440] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:03:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:03:44.238252 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498aeb8-6dbd-4d37-9664-1a6d5379f42c None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:44.238771 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498aeb8-6dbd-4d37-9664-1a6d5379f42c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:44.238771 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1498aeb8-6dbd-4d37-9664-1a6d5379f42c None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:44.240215 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 15:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:03:44.245923 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fea109-f5d4-4714-86e7-02f3b9a74dc1 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:44.246412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fea109-f5d4-4714-86e7-02f3b9a74dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:44.246526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fea109-f5d4-4714-86e7-02f3b9a74dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:44.289399 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:44.317237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-04fea109-f5d4-4714-86e7-02f3b9a74dc1 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:44.318497 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-04fea109-f5d4-4714-86e7-02f3b9a74dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:44.419671 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 15:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:03:44.459565 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498aeb8-6dbd-4d37-9664-1a6d5379f42c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:03:44.461074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1498aeb8-6dbd-4d37-9664-1a6d5379f42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:44.575567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1498aeb8-6dbd-4d37-9664-1a6d5379f42c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:03:44.616200 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:44.668427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce31a17-0bed-44ef-ab0e-3b61a6c5cec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:03:44.669098 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce31a17-0bed-44ef-ab0e-3b61a6c5cec5 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:44.669407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce31a17-0bed-44ef-ab0e-3b61a6c5cec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:44.669681 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce31a17-0bed-44ef-ab0e-3b61a6c5cec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:44.672044 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3ce31a17-0bed-44ef-ab0e-3b61a6c5cec5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:44.801163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce31a17-0bed-44ef-ab0e-3b61a6c5cec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:03:44.802631 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce31a17-0bed-44ef-ab0e-3b61a6c5cec5 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:03:44.822906 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 721/1443] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:03:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:03:47.093057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0287f9-de7a-41ef-b7ae-ae35f112ce20 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:47.093627 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0287f9-de7a-41ef-b7ae-ae35f112ce20 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:47.093627 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0287f9-de7a-41ef-b7ae-ae35f112ce20 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:47.095427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 15:03:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:03:47.101313 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1884275-bf7f-4d64-98fb-df92b1e931b1 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:47.101807 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1884275-bf7f-4d64-98fb-df92b1e931b1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:47.101927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1884275-bf7f-4d64-98fb-df92b1e931b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:47.167622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d1884275-bf7f-4d64-98fb-df92b1e931b1 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:47.168757 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d1884275-bf7f-4d64-98fb-df92b1e931b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:47.183433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:47.272523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 15:03:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:03:47.311760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0287f9-de7a-41ef-b7ae-ae35f112ce20 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:03:47.313305 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3e0287f9-de7a-41ef-b7ae-ae35f112ce20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:47.428023 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0287f9-de7a-41ef-b7ae-ae35f112ce20 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:03:47.440764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:47.505321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db182859-f3e3-4b50-aa08-9bf0c53e7db0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:03:47.505953 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db182859-f3e3-4b50-aa08-9bf0c53e7db0 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:47.506228 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db182859-f3e3-4b50-aa08-9bf0c53e7db0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:47.506498 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db182859-f3e3-4b50-aa08-9bf0c53e7db0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:47.508153 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-db182859-f3e3-4b50-aa08-9bf0c53e7db0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:47.617940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db182859-f3e3-4b50-aa08-9bf0c53e7db0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:03:47.619419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db182859-f3e3-4b50-aa08-9bf0c53e7db0 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:03:47.638358 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 724/1446] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:03:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:03:50.164317 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492fd54-97c2-41fc-bd2e-c5a9355e9a90 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:50.164850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492fd54-97c2-41fc-bd2e-c5a9355e9a90 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:50.164850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492fd54-97c2-41fc-bd2e-c5a9355e9a90 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:50.166159 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 15:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:03:50.172984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ff9df-e75c-44e9-921a-cad8e33ac07c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:50.173674 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ff9df-e75c-44e9-921a-cad8e33ac07c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:50.173952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ff9df-e75c-44e9-921a-cad8e33ac07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:50.223535 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:50.242921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-871ff9df-e75c-44e9-921a-cad8e33ac07c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:50.244144 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-871ff9df-e75c-44e9-921a-cad8e33ac07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:50.345377 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 15:03:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:03:50.385157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492fd54-97c2-41fc-bd2e-c5a9355e9a90 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:03:50.386693 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9492fd54-97c2-41fc-bd2e-c5a9355e9a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:50.495987 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492fd54-97c2-41fc-bd2e-c5a9355e9a90 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:03:50.537346 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:50.576641 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ccfc6-6f39-4aea-987c-b06c316cecfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:03:50.577787 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ccfc6-6f39-4aea-987c-b06c316cecfd service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:50.578077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ccfc6-6f39-4aea-987c-b06c316cecfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:50.578352 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ccfc6-6f39-4aea-987c-b06c316cecfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:50.580042 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-979ccfc6-6f39-4aea-987c-b06c316cecfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:50.692334 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ccfc6-6f39-4aea-987c-b06c316cecfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:03:50.693767 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ccfc6-6f39-4aea-987c-b06c316cecfd service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:03:50.713639 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 724/1449] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:03:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:03:52.887109 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc211f02-8b06-4a7d-863a-306f948b979a None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:52.887675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc211f02-8b06-4a7d-863a-306f948b979a None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:52.887675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc211f02-8b06-4a7d-863a-306f948b979a None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:52.889061 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 15:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 16 15:03:52.895540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897201f4-3405-40ef-a5dd-880ea17892e1 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:52.896052 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897201f4-3405-40ef-a5dd-880ea17892e1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:52.896161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897201f4-3405-40ef-a5dd-880ea17892e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:52.962938 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-897201f4-3405-40ef-a5dd-880ea17892e1 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:52.964163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-897201f4-3405-40ef-a5dd-880ea17892e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:52.986244 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:53.064973 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 15:03:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:03:53.104942 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc211f02-8b06-4a7d-863a-306f948b979a None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:03:53.106464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-bc211f02-8b06-4a7d-863a-306f948b979a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:53.211358 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc211f02-8b06-4a7d-863a-306f948b979a None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:03:53.231132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:53.286749 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c811f-edc8-472f-aa38-23e8e60f1213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:03:53.287427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c811f-edc8-472f-aa38-23e8e60f1213 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:53.287709 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c811f-edc8-472f-aa38-23e8e60f1213 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:53.287983 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c811f-edc8-472f-aa38-23e8e60f1213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:53.289628 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3e0c811f-edc8-472f-aa38-23e8e60f1213 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:53.401811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c811f-edc8-472f-aa38-23e8e60f1213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:03:53.403290 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c811f-edc8-472f-aa38-23e8e60f1213 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:03:53.422266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 727/1452] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:03:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:03:55.545232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95977e0a-2945-4d8b-9ead-df077574b7d6 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:55.546462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95977e0a-2945-4d8b-9ead-df077574b7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:55.546462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95977e0a-2945-4d8b-9ead-df077574b7d6 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:55.547738 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 15:03:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:03:55.553810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f34049-933d-40c2-8551-c4dd53ca962c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:55.554295 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f34049-933d-40c2-8551-c4dd53ca962c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:55.554402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f34049-933d-40c2-8551-c4dd53ca962c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:55.618285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:55.621177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-f2f34049-933d-40c2-8551-c4dd53ca962c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:55.622430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-f2f34049-933d-40c2-8551-c4dd53ca962c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:55.722417 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 15:03:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:03:55.760458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95977e0a-2945-4d8b-9ead-df077574b7d6 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:03:55.761980 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-95977e0a-2945-4d8b-9ead-df077574b7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:55.867524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95977e0a-2945-4d8b-9ead-df077574b7d6 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:03:55.918047 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:55.944214 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91832f03-ee3f-4a93-ac78-3e390008a1c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:03:55.944868 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91832f03-ee3f-4a93-ac78-3e390008a1c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:55.945151 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91832f03-ee3f-4a93-ac78-3e390008a1c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:55.945428 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91832f03-ee3f-4a93-ac78-3e390008a1c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:55.947109 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-91832f03-ee3f-4a93-ac78-3e390008a1c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:56.059467 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91832f03-ee3f-4a93-ac78-3e390008a1c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:03:56.060882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91832f03-ee3f-4a93-ac78-3e390008a1c4 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:03:56.083221 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 727/1455] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:03:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:03:58.258239 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a7d7c-6c29-4d8c-b525-767e5ea91b2b None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:58.258779 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a7d7c-6c29-4d8c-b525-767e5ea91b2b None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:58.258779 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a7d7c-6c29-4d8c-b525-767e5ea91b2b None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:58.260168 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 15:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:03:58.266752 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d2ca5-9142-488c-9b90-4d4fea2e7f96 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:58.267223 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d2ca5-9142-488c-9b90-4d4fea2e7f96 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:58.267348 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d2ca5-9142-488c-9b90-4d4fea2e7f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:58.459718 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-168d2ca5-9142-488c-9b90-4d4fea2e7f96 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:03:58.460927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-168d2ca5-9142-488c-9b90-4d4fea2e7f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:58.491704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:58.562291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 15:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:03:58.601416 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a7d7c-6c29-4d8c-b525-767e5ea91b2b None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:03:58.603163 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-756a7d7c-6c29-4d8c-b525-767e5ea91b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:58.705331 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a7d7c-6c29-4d8c-b525-767e5ea91b2b None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:03:58.734345 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:03:58.781381 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83604b01-f4c3-4245-98ad-9f43a0321e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:03:58.782025 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83604b01-f4c3-4245-98ad-9f43a0321e2b service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:03:58.782300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83604b01-f4c3-4245-98ad-9f43a0321e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:03:58.782566 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83604b01-f4c3-4245-98ad-9f43a0321e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:03:58.784218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-83604b01-f4c3-4245-98ad-9f43a0321e2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:03:58.893666 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83604b01-f4c3-4245-98ad-9f43a0321e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:03:58.895158 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83604b01-f4c3-4245-98ad-9f43a0321e2b service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:03:58.915397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 730/1458] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:03:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:04:01.001628 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf33e2-2733-48e3-8a91-102282360cb4 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:01.002166 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf33e2-2733-48e3-8a91-102282360cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:01.002292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf33e2-2733-48e3-8a91-102282360cb4 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:01.003706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 15:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:04:01.009364 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1f5dc-ae3e-48fa-8222-55db0f45ffe9 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:01.009830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1f5dc-ae3e-48fa-8222-55db0f45ffe9 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:01.009937 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1f5dc-ae3e-48fa-8222-55db0f45ffe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:01.077431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-35b1f5dc-ae3e-48fa-8222-55db0f45ffe9 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:04:01.078655 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-35b1f5dc-ae3e-48fa-8222-55db0f45ffe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:01.087313 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:01.176425 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 15:04:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:04:01.214596 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf33e2-2733-48e3-8a91-102282360cb4 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:04:01.216131 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4acf33e2-2733-48e3-8a91-102282360cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:01.320666 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf33e2-2733-48e3-8a91-102282360cb4 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:04:01.376893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:01.395419 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e835762-e0a5-4e9e-aaa1-7721052220cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:04:01.396069 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e835762-e0a5-4e9e-aaa1-7721052220cd service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:01.396351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e835762-e0a5-4e9e-aaa1-7721052220cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:01.396624 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e835762-e0a5-4e9e-aaa1-7721052220cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:01.398257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5e835762-e0a5-4e9e-aaa1-7721052220cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:01.510504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e835762-e0a5-4e9e-aaa1-7721052220cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:04:01.511892 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e835762-e0a5-4e9e-aaa1-7721052220cd service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:04:01.531194 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 730/1461] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:04:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:04:03.646712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab8cdf-a917-46a7-a57a-5f6d88d8b477 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:03.647263 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab8cdf-a917-46a7-a57a-5f6d88d8b477 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:03.647361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab8cdf-a917-46a7-a57a-5f6d88d8b477 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:03.648706 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 15:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:04:03.654343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9dfdd8-9b66-4ede-81cd-3eb24c48829f None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:03.655436 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9dfdd8-9b66-4ede-81cd-3eb24c48829f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:03.655436 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9dfdd8-9b66-4ede-81cd-3eb24c48829f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:03.720685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-bf9dfdd8-9b66-4ede-81cd-3eb24c48829f None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:04:03.721911 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bf9dfdd8-9b66-4ede-81cd-3eb24c48829f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:03.760015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:03.818073 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 15:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:04:03.857434 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab8cdf-a917-46a7-a57a-5f6d88d8b477 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:04:03.859168 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-6dab8cdf-a917-46a7-a57a-5f6d88d8b477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:03.964374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab8cdf-a917-46a7-a57a-5f6d88d8b477 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:04:04.002142 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:04.039512 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517028b9-c4ca-4842-a29f-82364fcbe812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:04:04.040163 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517028b9-c4ca-4842-a29f-82364fcbe812 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:04.040450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517028b9-c4ca-4842-a29f-82364fcbe812 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:04.040728 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517028b9-c4ca-4842-a29f-82364fcbe812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:04.042382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-517028b9-c4ca-4842-a29f-82364fcbe812 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:04.159210 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517028b9-c4ca-4842-a29f-82364fcbe812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:04:04.160613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517028b9-c4ca-4842-a29f-82364fcbe812 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:04:04.180592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 733/1464] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:04:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:04:06.396490 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a20839-98a6-46d9-a7d4-10bff63d3769 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:06.397100 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a20839-98a6-46d9-a7d4-10bff63d3769 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:06.397100 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a20839-98a6-46d9-a7d4-10bff63d3769 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:06.398400 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 15:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:04:06.403942 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015234d-bedc-4380-8a93-efdd0588e75c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:06.404451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015234d-bedc-4380-8a93-efdd0588e75c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:06.404616 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015234d-bedc-4380-8a93-efdd0588e75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:06.474959 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-9015234d-bedc-4380-8a93-efdd0588e75c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:04:06.476385 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-9015234d-bedc-4380-8a93-efdd0588e75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:06.494658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:06.584551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 15:04:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:04:06.620374 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a20839-98a6-46d9-a7d4-10bff63d3769 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:04:06.621899 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-29a20839-98a6-46d9-a7d4-10bff63d3769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:06.743592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a20839-98a6-46d9-a7d4-10bff63d3769 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:04:06.810536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:06.818587 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869b639-50d6-4536-b97a-716b8a01ee33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:04:06.819266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869b639-50d6-4536-b97a-716b8a01ee33 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:06.819548 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869b639-50d6-4536-b97a-716b8a01ee33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:06.819840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869b639-50d6-4536-b97a-716b8a01ee33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:06.821481 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3869b639-50d6-4536-b97a-716b8a01ee33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:06.939233 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869b639-50d6-4536-b97a-716b8a01ee33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:04:06.940675 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869b639-50d6-4536-b97a-716b8a01ee33 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:04:06.959504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 733/1467] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:04:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:04:09.035883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b64a28-8896-4549-845d-0810e789ea2f None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:09.036445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b64a28-8896-4549-845d-0810e789ea2f None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:09.036445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b64a28-8896-4549-845d-0810e789ea2f None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:09.037797 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 15:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:04:09.043368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7adbc-8bc6-42c4-9e42-20c10f208549 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:09.043882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7adbc-8bc6-42c4-9e42-20c10f208549 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:09.043993 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7adbc-8bc6-42c4-9e42-20c10f208549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:09.117822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5dd7adbc-8bc6-42c4-9e42-20c10f208549 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:04:09.119089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5dd7adbc-8bc6-42c4-9e42-20c10f208549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:09.171344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:09.223609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 15:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:04:09.260730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b64a28-8896-4549-845d-0810e789ea2f None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:04:09.263690 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-16b64a28-8896-4549-845d-0810e789ea2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:09.372444 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b64a28-8896-4549-845d-0810e789ea2f None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:04:09.423237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:09.450077 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea207bf0-ed3a-41c6-85ff-341cb4adb5f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:04:09.450722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea207bf0-ed3a-41c6-85ff-341cb4adb5f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:09.451139 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea207bf0-ed3a-41c6-85ff-341cb4adb5f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:09.452269 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea207bf0-ed3a-41c6-85ff-341cb4adb5f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:09.454698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ea207bf0-ed3a-41c6-85ff-341cb4adb5f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:09.564381 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea207bf0-ed3a-41c6-85ff-341cb4adb5f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:04:09.565980 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea207bf0-ed3a-41c6-85ff-341cb4adb5f0 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:04:09.585701 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 736/1470] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:04:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:04:19.221748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=22 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:19.583341 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=66 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:29.450359 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9932c0-aa76-497d-8acd-11f002e1d3d2 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:29.451462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9932c0-aa76-497d-8acd-11f002e1d3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:29.451578 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9932c0-aa76-497d-8acd-11f002e1d3d2 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:29.453528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 735/1471] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:04:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:04:29.458788 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d71d89-0857-4795-b60d-f708d2f7c720 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:29.459207 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d71d89-0857-4795-b60d-f708d2f7c720 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:29.459392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d71d89-0857-4795-b60d-f708d2f7c720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:29.525322 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-68d71d89-0857-4795-b60d-f708d2f7c720 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:04:29.526445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-68d71d89-0857-4795-b60d-f708d2f7c720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:29.625422 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 737/1472] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:04:29.674596 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe80f9-602c-4320-9280-09213a187721 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:29.675052 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe80f9-602c-4320-9280-09213a187721 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:29.675393 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe80f9-602c-4320-9280-09213a187721 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:29.676910 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 736/1473] 10.222.0.27 () {54 vars in 960 bytes} [Sat Jul 16 15:04:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:04:29.682087 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7840d3b-fd16-4c9b-af0e-3e196ab5c0c6 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:29.682374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7840d3b-fd16-4c9b-af0e-3e196ab5c0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:29.682649 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7840d3b-fd16-4c9b-af0e-3e196ab5c0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:29.758402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-a7840d3b-fd16-4c9b-af0e-3e196ab5c0c6 None None] MFA Rules not processed for user `7d689162291f4f4b97e886ea20d3a8b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:04:29.759647 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a7840d3b-fd16-4c9b-af0e-3e196ab5c0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:29.788910 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:29.839270 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 738/1474] 10.222.0.27 () {58 vars in 1057 bytes} [Sat Jul 16 15:04:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:04:29.847593 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe80f9-602c-4320-9280-09213a187721 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:04:29.849086 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-18fe80f9-602c-4320-9280-09213a187721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:29.962122 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe80f9-602c-4320-9280-09213a187721 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:04:30.039624 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01b8ce-345b-40ab-8714-6b23ba027eb0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d689162291f4f4b97e886ea20d3a8b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:04:30.040177 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01b8ce-345b-40ab-8714-6b23ba027eb0 service glance] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:30.040489 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01b8ce-345b-40ab-8714-6b23ba027eb0 service glance] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:30.040793 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01b8ce-345b-40ab-8714-6b23ba027eb0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:30.042453 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-da01b8ce-345b-40ab-8714-6b23ba027eb0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:30.099647 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:30.159021 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01b8ce-345b-40ab-8714-6b23ba027eb0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:04:30.160493 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01b8ce-345b-40ab-8714-6b23ba027eb0 service glance] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:04:30.180152 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 737/1475] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:04:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:04:32.379844 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761227a2-f24c-4182-bd8f-5d7c1c7a6ebd None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:32.380509 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761227a2-f24c-4182-bd8f-5d7c1c7a6ebd None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:32.380509 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761227a2-f24c-4182-bd8f-5d7c1c7a6ebd None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:32.381831 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 739/1476] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:04:32.387670 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21724393-b4b2-475c-9434-45d2b6cf9ef2 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:32.388161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21724393-b4b2-475c-9434-45d2b6cf9ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:32.388335 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21724393-b4b2-475c-9434-45d2b6cf9ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:32.460657 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-21724393-b4b2-475c-9434-45d2b6cf9ef2 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:04:32.461874 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-21724393-b4b2-475c-9434-45d2b6cf9ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:32.559064 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 738/1477] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:04:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:04:32.584101 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761227a2-f24c-4182-bd8f-5d7c1c7a6ebd None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:04:32.585623 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-761227a2-f24c-4182-bd8f-5d7c1c7a6ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:32.709510 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761227a2-f24c-4182-bd8f-5d7c1c7a6ebd None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:04:32.759919 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:32.805190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b8a6b-04cc-43f5-b702-d74585562c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:04:32.805861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b8a6b-04cc-43f5-b702-d74585562c89 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:32.806320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b8a6b-04cc-43f5-b702-d74585562c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:32.806837 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b8a6b-04cc-43f5-b702-d74585562c89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:32.807711 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b8a6b-04cc-43f5-b702-d74585562c89 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:04:32.809796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: WARNING py.warnings [None req-b42b8a6b-04cc-43f5-b702-d74585562c89 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 15:04:32.809796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: warnings.warn(msg) Jul 16 15:04:32.809796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: Jul 16 15:04:32.811784 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b8a6b-04cc-43f5-b702-d74585562c89 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:04:32.832302 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 740/1478] 10.222.0.27 () {56 vars in 1108 bytes} [Sat Jul 16 15:04:32 2022] GET /identity/v3/projects => generated 2553 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:04:34.907148 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8274d5c-47ec-496d-91c2-1de7e4cf315e None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:34.907624 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8274d5c-47ec-496d-91c2-1de7e4cf315e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:34.907732 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8274d5c-47ec-496d-91c2-1de7e4cf315e None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:34.909114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 739/1479] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:04:34.914631 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82434383-6482-48d1-92b9-98e864c8060d None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:34.915123 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82434383-6482-48d1-92b9-98e864c8060d None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:34.915219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82434383-6482-48d1-92b9-98e864c8060d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:34.983554 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-82434383-6482-48d1-92b9-98e864c8060d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:04:34.984771 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-82434383-6482-48d1-92b9-98e864c8060d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:35.086129 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 741/1480] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:04:35.130534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8274d5c-47ec-496d-91c2-1de7e4cf315e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:04:35.132093 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a8274d5c-47ec-496d-91c2-1de7e4cf315e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:35.141823 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:35.282326 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8274d5c-47ec-496d-91c2-1de7e4cf315e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:04:35.362829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:35.395866 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9d94d-50a9-43b1-b20b-a4445665501d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:04:35.396831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9d94d-50a9-43b1-b20b-a4445665501d service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:35.397124 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9d94d-50a9-43b1-b20b-a4445665501d service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:35.397400 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9d94d-50a9-43b1-b20b-a4445665501d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:35.399081 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-98c9d94d-50a9-43b1-b20b-a4445665501d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:35.488038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9d94d-50a9-43b1-b20b-a4445665501d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:04:35.489634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9d94d-50a9-43b1-b20b-a4445665501d service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:04:35.508041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 740/1481] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:04:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:04:37.745404 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a033d-52f3-422d-9e23-b6c2003e3133 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:37.746035 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a033d-52f3-422d-9e23-b6c2003e3133 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:37.746035 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a033d-52f3-422d-9e23-b6c2003e3133 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:37.747400 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 742/1482] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:04:37.752921 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08243d82-3e55-425a-a8a2-9f2487668468 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:37.753376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08243d82-3e55-425a-a8a2-9f2487668468 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:37.753490 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08243d82-3e55-425a-a8a2-9f2487668468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:37.817584 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-08243d82-3e55-425a-a8a2-9f2487668468 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:04:37.818798 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-08243d82-3e55-425a-a8a2-9f2487668468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:37.866246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:37.922193 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 741/1483] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:04:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:04:37.959830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a033d-52f3-422d-9e23-b6c2003e3133 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:04:37.961383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-655a033d-52f3-422d-9e23-b6c2003e3133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:37.985616 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:38.067013 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a033d-52f3-422d-9e23-b6c2003e3133 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:04:38.142317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72172d2-ff73-4161-85c5-49649d6ff2a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:04:38.142985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72172d2-ff73-4161-85c5-49649d6ff2a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:38.143323 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72172d2-ff73-4161-85c5-49649d6ff2a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:38.143544 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72172d2-ff73-4161-85c5-49649d6ff2a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:38.145192 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-d72172d2-ff73-4161-85c5-49649d6ff2a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:38.220198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:38.255360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72172d2-ff73-4161-85c5-49649d6ff2a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:04:38.256830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72172d2-ff73-4161-85c5-49649d6ff2a6 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:04:38.275725 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 743/1484] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:04:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:04:40.381856 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70f234-3fb5-4963-8a07-e3b44b26d49b None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:40.382587 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70f234-3fb5-4963-8a07-e3b44b26d49b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:40.382587 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70f234-3fb5-4963-8a07-e3b44b26d49b None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:40.384122 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 742/1485] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:04:40.390699 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86c59c-9a2c-4d53-b354-5b739ee32c88 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:40.391320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86c59c-9a2c-4d53-b354-5b739ee32c88 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:40.391320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86c59c-9a2c-4d53-b354-5b739ee32c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:40.471131 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-7d86c59c-9a2c-4d53-b354-5b739ee32c88 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:04:40.472363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7d86c59c-9a2c-4d53-b354-5b739ee32c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:40.574481 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 744/1486] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:04:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:04:40.613727 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70f234-3fb5-4963-8a07-e3b44b26d49b None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:04:40.615302 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1a70f234-3fb5-4963-8a07-e3b44b26d49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:40.720603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70f234-3fb5-4963-8a07-e3b44b26d49b None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:04:40.795396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:40.797255 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8203785-39af-47c5-8c69-c6e232106712 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:04:40.797906 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8203785-39af-47c5-8c69-c6e232106712 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:40.798187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8203785-39af-47c5-8c69-c6e232106712 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:40.798461 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8203785-39af-47c5-8c69-c6e232106712 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:40.800133 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e8203785-39af-47c5-8c69-c6e232106712 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:40.915105 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8203785-39af-47c5-8c69-c6e232106712 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:04:40.916599 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8203785-39af-47c5-8c69-c6e232106712 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:04:40.935311 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 743/1487] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:04:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:04:43.147925 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152d494-6c64-4559-b53f-fec3a1c7749c None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:43.148534 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152d494-6c64-4559-b53f-fec3a1c7749c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:43.148534 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152d494-6c64-4559-b53f-fec3a1c7749c None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:43.149955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 745/1488] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:04:43.155732 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2ceaf-633c-45a0-bf89-c2d53fc50ee0 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:43.156230 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2ceaf-633c-45a0-bf89-c2d53fc50ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:43.156230 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2ceaf-633c-45a0-bf89-c2d53fc50ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:43.227643 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a1f2ceaf-633c-45a0-bf89-c2d53fc50ee0 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:04:43.228893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a1f2ceaf-633c-45a0-bf89-c2d53fc50ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:43.287893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:43.335646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 744/1489] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:04:43.376814 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152d494-6c64-4559-b53f-fec3a1c7749c None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:04:43.378361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e152d494-6c64-4559-b53f-fec3a1c7749c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:43.415868 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:43.485559 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152d494-6c64-4559-b53f-fec3a1c7749c None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:04:43.563078 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9d9ef-071c-41b5-9797-bd0299ebde3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:04:43.563730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9d9ef-071c-41b5-9797-bd0299ebde3e service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:43.564009 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9d9ef-071c-41b5-9797-bd0299ebde3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:43.564277 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9d9ef-071c-41b5-9797-bd0299ebde3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:43.565970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-01a9d9ef-071c-41b5-9797-bd0299ebde3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:43.648307 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:43.680830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9d9ef-071c-41b5-9797-bd0299ebde3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:04:43.682167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9d9ef-071c-41b5-9797-bd0299ebde3e service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:04:43.701280 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 746/1490] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:04:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:04:46.189953 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0bd36-a227-4c8a-9a59-d23c53bc96bb None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:46.190507 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0bd36-a227-4c8a-9a59-d23c53bc96bb None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:46.190507 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0bd36-a227-4c8a-9a59-d23c53bc96bb None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:46.191976 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 745/1491] 10.222.0.27 () {54 vars in 926 bytes} [Sat Jul 16 15:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:04:46.200079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c719e-f436-4260-9497-e9d8d9e9d94d None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:46.200633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c719e-f436-4260-9497-e9d8d9e9d94d None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:46.200633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c719e-f436-4260-9497-e9d8d9e9d94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:46.274319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-a13c719e-f436-4260-9497-e9d8d9e9d94d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:04:46.275587 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a13c719e-f436-4260-9497-e9d8d9e9d94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:46.379153 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 747/1492] 10.222.0.27 () {58 vars in 1023 bytes} [Sat Jul 16 15:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:04:46.424864 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0bd36-a227-4c8a-9a59-d23c53bc96bb None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:04:46.426430 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e3d0bd36-a227-4c8a-9a59-d23c53bc96bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:46.539812 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0bd36-a227-4c8a-9a59-d23c53bc96bb None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:04:46.617208 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85932115-34f3-4d24-bae1-3a05a6660f7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:04:46.617859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85932115-34f3-4d24-bae1-3a05a6660f7c service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:04:46.618135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85932115-34f3-4d24-bae1-3a05a6660f7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:04:46.618401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85932115-34f3-4d24-bae1-3a05a6660f7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:04:46.620082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-85932115-34f3-4d24-bae1-3a05a6660f7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:04:46.630023 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:46.738439 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85932115-34f3-4d24-bae1-3a05a6660f7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:04:46.739936 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85932115-34f3-4d24-bae1-3a05a6660f7c service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:04:46.759442 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 746/1493] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:04:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:04:56.377159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=82 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:04:56.756924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=50 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:06:51.948909 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b230e-6c5b-4f48-959c-feb5193a6d7e None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:06:51.949499 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b230e-6c5b-4f48-959c-feb5193a6d7e None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:06:51.949499 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b230e-6c5b-4f48-959c-feb5193a6d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:06:52.029104 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-738b230e-6c5b-4f48-959c-feb5193a6d7e None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:06:52.030225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-738b230e-6c5b-4f48-959c-feb5193a6d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:06:52.139968 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 748/1494] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:06:52.149045 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e35d3-d517-403c-8fba-f4d9bb474ba0 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:06:52.149592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e35d3-d517-403c-8fba-f4d9bb474ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:06:52.149592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e35d3-d517-403c-8fba-f4d9bb474ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:06:52.169145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-461e35d3-d517-403c-8fba-f4d9bb474ba0 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:06:52.170364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-461e35d3-d517-403c-8fba-f4d9bb474ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:06:52.179558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 747/1495] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:06:52.184340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b230e-6c5b-4f48-959c-feb5193a6d7e None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:06:52.194392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b230e-6c5b-4f48-959c-feb5193a6d7e None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:06:52.199390 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff6df4-50f2-4ea1-8754-23c46d7f7a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:06:52.200047 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff6df4-50f2-4ea1-8754-23c46d7f7a44 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:06:52.200332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff6df4-50f2-4ea1-8754-23c46d7f7a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:06:52.200611 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff6df4-50f2-4ea1-8754-23c46d7f7a44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:06:52.201347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff6df4-50f2-4ea1-8754-23c46d7f7a44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:06:52.201656 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff6df4-50f2-4ea1-8754-23c46d7f7a44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:06:52.202675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: WARNING py.warnings [None req-f1ff6df4-50f2-4ea1-8754-23c46d7f7a44 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 15:06:52.202675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: warnings.warn(msg) Jul 16 15:06:52.202675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: Jul 16 15:06:52.204118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff6df4-50f2-4ea1-8754-23c46d7f7a44 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:06:52.216681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 749/1496] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:06:52.226189 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e35d3-d517-403c-8fba-f4d9bb474ba0 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:06:52.236163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e35d3-d517-403c-8fba-f4d9bb474ba0 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:06:52.241300 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5bdb94-f788-4cb3-946c-3bbe1a3eadc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:06:52.241966 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5bdb94-f788-4cb3-946c-3bbe1a3eadc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:06:52.242245 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5bdb94-f788-4cb3-946c-3bbe1a3eadc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:06:52.242519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5bdb94-f788-4cb3-946c-3bbe1a3eadc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:06:52.243252 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5bdb94-f788-4cb3-946c-3bbe1a3eadc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:06:52.245017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: WARNING py.warnings [None req-cf5bdb94-f788-4cb3-946c-3bbe1a3eadc8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 15:06:52.245017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: warnings.warn(msg) Jul 16 15:06:52.245017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: Jul 16 15:06:52.246207 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5bdb94-f788-4cb3-946c-3bbe1a3eadc8 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:06:52.397005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 748/1497] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:06:52 2022] POST /identity/v3/projects => generated 379 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:06:52.401137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff6df4-50f2-4ea1-8754-23c46d7f7a44 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:06:52.411383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff6df4-50f2-4ea1-8754-23c46d7f7a44 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:06:52.467708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:06:52.474591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b116ff-db6c-4b44-a05e-9fa81c923cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:06:52.478613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b116ff-db6c-4b44-a05e-9fa81c923cd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:06:52.478613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b116ff-db6c-4b44-a05e-9fa81c923cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:06:52.478613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b116ff-db6c-4b44-a05e-9fa81c923cd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:06:52.478613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b116ff-db6c-4b44-a05e-9fa81c923cd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:06:52.478613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: WARNING py.warnings [None req-c7b116ff-db6c-4b44-a05e-9fa81c923cd3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 15:06:52.478613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: warnings.warn(msg) Jul 16 15:06:52.478613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: Jul 16 15:06:52.479271 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b116ff-db6c-4b44-a05e-9fa81c923cd3 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:06:52.578578 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 750/1498] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:06:52 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:06:52.582748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5bdb94-f788-4cb3-946c-3bbe1a3eadc8 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:06:52.593005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5bdb94-f788-4cb3-946c-3bbe1a3eadc8 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:06:52.598364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d643b1f-81dc-4962-b31e-ededce7e8536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:06:52.598511 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d643b1f-81dc-4962-b31e-ededce7e8536 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:06:52.598791 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d643b1f-81dc-4962-b31e-ededce7e8536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:06:52.599100 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d643b1f-81dc-4962-b31e-ededce7e8536 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:06:52.599828 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d643b1f-81dc-4962-b31e-ededce7e8536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:06:52.600840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: WARNING py.warnings [None req-4d643b1f-81dc-4962-b31e-ededce7e8536 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 15:06:52.600840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: warnings.warn(msg) Jul 16 15:06:52.600840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: Jul 16 15:06:52.601706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d643b1f-81dc-4962-b31e-ededce7e8536 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:06:52.615188 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 749/1499] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:06:52.619735 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b116ff-db6c-4b44-a05e-9fa81c923cd3 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:06:52.639526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b116ff-db6c-4b44-a05e-9fa81c923cd3 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:06:52.645257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e225504-10dc-4d12-9444-4d1104e7bb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:06:52.645929 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e225504-10dc-4d12-9444-4d1104e7bb56 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:06:52.646213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e225504-10dc-4d12-9444-4d1104e7bb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:06:52.646485 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e225504-10dc-4d12-9444-4d1104e7bb56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:06:52.647305 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e225504-10dc-4d12-9444-4d1104e7bb56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:06:52.648305 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: WARNING py.warnings [None req-8e225504-10dc-4d12-9444-4d1104e7bb56 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 15:06:52.648305 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: warnings.warn(msg) Jul 16 15:06:52.648305 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: Jul 16 15:06:52.649202 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e225504-10dc-4d12-9444-4d1104e7bb56 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:06:52.663289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 751/1500] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:06:52.667612 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d643b1f-81dc-4962-b31e-ededce7e8536 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:06:52.678162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d643b1f-81dc-4962-b31e-ededce7e8536 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:06:52.682779 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ca575-609a-4a02-98d2-95ef52d1b13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:06:52.683706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ca575-609a-4a02-98d2-95ef52d1b13c admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:06:52.683987 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ca575-609a-4a02-98d2-95ef52d1b13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:06:52.684264 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ca575-609a-4a02-98d2-95ef52d1b13c admin admin] PATH_INFO: `/v3/projects/fdf48ef9191942658cb9984b3b86415d/users/f8bf06173311446dabe1c675e9dcbc50/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:06:52.700688 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ca575-609a-4a02-98d2-95ef52d1b13c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf48ef9191942658cb9984b3b86415d, user_id=f8bf06173311446dabe1c675e9dcbc50, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:06:52.701725 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: WARNING py.warnings [None req-cb0ca575-609a-4a02-98d2-95ef52d1b13c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 15:06:52.701725 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: warnings.warn(msg) Jul 16 15:06:52.701725 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: Jul 16 15:06:52.704042 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ca575-609a-4a02-98d2-95ef52d1b13c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:06:52.777640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 750/1501] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:06:52 2022] PUT /identity/v3/projects/fdf48ef9191942658cb9984b3b86415d/users/f8bf06173311446dabe1c675e9dcbc50/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:06:52.790063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676641b3-8392-4c2c-ad76-168341bb1cc5 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:06:52.798867 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676641b3-8392-4c2c-ad76-168341bb1cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:06:52.798867 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676641b3-8392-4c2c-ad76-168341bb1cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:06:52.822551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-676641b3-8392-4c2c-ad76-168341bb1cc5 None None] MFA Rules not processed for user `f8bf06173311446dabe1c675e9dcbc50`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:06:52.824101 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-676641b3-8392-4c2c-ad76-168341bb1cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:06:52.889354 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 752/1502] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:06:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:06:52.902941 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04dd68-a023-40df-8259-8df605c2e49d None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:06:52.903068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04dd68-a023-40df-8259-8df605c2e49d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:06:52.903331 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff04dd68-a023-40df-8259-8df605c2e49d None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:06:52.904691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 751/1503] 10.222.0.27 () {54 vars in 963 bytes} [Sat Jul 16 15:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:06:52.909945 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573ea9e-7f76-466b-a9b1-7a9b0e2228c9 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:06:52.910234 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573ea9e-7f76-466b-a9b1-7a9b0e2228c9 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:06:52.910512 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573ea9e-7f76-466b-a9b1-7a9b0e2228c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:06:52.973828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-5573ea9e-7f76-466b-a9b1-7a9b0e2228c9 None None] MFA Rules not processed for user `8c21d12c977e4c36b4f0ed442b11ef49`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:06:52.975081 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5573ea9e-7f76-466b-a9b1-7a9b0e2228c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:06:53.054421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 753/1504] 10.222.0.27 () {58 vars in 1060 bytes} [Sat Jul 16 15:06:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:06:53.062318 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04dd68-a023-40df-8259-8df605c2e49d None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:06:53.073760 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff04dd68-a023-40df-8259-8df605c2e49d None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:06:53.077482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a631a-40ac-4f39-8cd9-89f57bf45a3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:06:53.078142 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a631a-40ac-4f39-8cd9-89f57bf45a3b service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:06:53.078424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a631a-40ac-4f39-8cd9-89f57bf45a3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:06:53.078697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a631a-40ac-4f39-8cd9-89f57bf45a3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:06:53.087858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a631a-40ac-4f39-8cd9-89f57bf45a3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:06:53.089309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a631a-40ac-4f39-8cd9-89f57bf45a3b service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:06:53.099938 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 752/1505] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:06:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:06:53.144289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa487b3-df32-440a-8f25-dd837580b3aa None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:06:53.144573 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa487b3-df32-440a-8f25-dd837580b3aa None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:06:53.144846 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa487b3-df32-440a-8f25-dd837580b3aa None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:06:53.146398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 754/1506] 10.222.0.27 () {52 vars in 829 bytes} [Sat Jul 16 15:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:06:53.149909 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a631a-40ac-4f39-8cd9-89f57bf45a3b service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:06:53.215208 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a631a-40ac-4f39-8cd9-89f57bf45a3b service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:06:53.220966 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b535d-863c-4a9f-98dd-2926191b469d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:06:53.222000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b535d-863c-4a9f-98dd-2926191b469d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:06:53.222353 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b535d-863c-4a9f-98dd-2926191b469d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:06:53.222704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b535d-863c-4a9f-98dd-2926191b469d admin admin] PATH_INFO: `/v3/users/f8bf06173311446dabe1c675e9dcbc50` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:06:53.224051 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b535d-863c-4a9f-98dd-2926191b469d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8bf06173311446dabe1c675e9dcbc50)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:06:53.225267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: WARNING py.warnings [None req-c88b535d-863c-4a9f-98dd-2926191b469d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 15:06:53.225267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: warnings.warn(msg) Jul 16 15:06:53.225267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: Jul 16 15:06:53.226585 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b535d-863c-4a9f-98dd-2926191b469d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:06:53.430724 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c88b535d-863c-4a9f-98dd-2926191b469d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8bf06173311446dabe1c675e9dcbc50'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:06:53.495444 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c88b535d-863c-4a9f-98dd-2926191b469d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8bf06173311446dabe1c675e9dcbc50'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:06:53.496009 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c88b535d-863c-4a9f-98dd-2926191b469d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8bf06173311446dabe1c675e9dcbc50'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:06:53.496720 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c88b535d-863c-4a9f-98dd-2926191b469d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8bf06173311446dabe1c675e9dcbc50'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:06:53.511822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c88b535d-863c-4a9f-98dd-2926191b469d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8bf06173311446dabe1c675e9dcbc50'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:06:53.528899 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:06:53.537975 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 753/1507] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:06:53 2022] DELETE /identity/v3/users/f8bf06173311446dabe1c675e9dcbc50 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:06:53.542694 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c527d-5f05-4333-9db5-6e1383b42d66 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:06:53.543015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c527d-5f05-4333-9db5-6e1383b42d66 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:06:53.543340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554c527d-5f05-4333-9db5-6e1383b42d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:06:53.562365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-554c527d-5f05-4333-9db5-6e1383b42d66 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:06:53.564383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-554c527d-5f05-4333-9db5-6e1383b42d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:06:53.574812 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:06:53.626629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 755/1508] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:06:53.636337 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b535d-863c-4a9f-98dd-2926191b469d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:06:53.637768 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c88b535d-863c-4a9f-98dd-2926191b469d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:06:53.720215 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b535d-863c-4a9f-98dd-2926191b469d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:06:53.724018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade2ff3-3fff-43f0-af6b-7a8ee6bbea40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:06:53.724671 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade2ff3-3fff-43f0-af6b-7a8ee6bbea40 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:06:53.724952 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade2ff3-3fff-43f0-af6b-7a8ee6bbea40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:06:53.725227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade2ff3-3fff-43f0-af6b-7a8ee6bbea40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:06:53.735657 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade2ff3-3fff-43f0-af6b-7a8ee6bbea40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:06:53.737114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade2ff3-3fff-43f0-af6b-7a8ee6bbea40 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:06:53.747388 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 754/1509] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:06:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:06:54.271429 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c527d-5f05-4333-9db5-6e1383b42d66 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:06:54.272998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-554c527d-5f05-4333-9db5-6e1383b42d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:06:54.294598 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554c527d-5f05-4333-9db5-6e1383b42d66 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:06:54.300478 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa387f4-0845-4e16-8907-02cc70b4b938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:06:54.301440 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa387f4-0845-4e16-8907-02cc70b4b938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:06:54.301715 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa387f4-0845-4e16-8907-02cc70b4b938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:06:54.302012 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa387f4-0845-4e16-8907-02cc70b4b938 admin admin] PATH_INFO: `/v3/projects/fdf48ef9191942658cb9984b3b86415d` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:06:54.303325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa387f4-0845-4e16-8907-02cc70b4b938 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdf48ef9191942658cb9984b3b86415d)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:06:54.306213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: WARNING py.warnings [None req-faa387f4-0845-4e16-8907-02cc70b4b938 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 15:06:54.306213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: warnings.warn(msg) Jul 16 15:06:54.306213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: Jul 16 15:06:54.306213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa387f4-0845-4e16-8907-02cc70b4b938 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:06:54.475814 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-faa387f4-0845-4e16-8907-02cc70b4b938 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdf48ef9191942658cb9984b3b86415d', 'request_id': 'req-faa387f4-0845-4e16-8907-02cc70b4b938'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:06:54.487634 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-faa387f4-0845-4e16-8907-02cc70b4b938 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdf48ef9191942658cb9984b3b86415d', 'request_id': 'req-faa387f4-0845-4e16-8907-02cc70b4b938'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:06:54.507004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-faa387f4-0845-4e16-8907-02cc70b4b938 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdf48ef9191942658cb9984b3b86415d', 'request_id': 'req-faa387f4-0845-4e16-8907-02cc70b4b938'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:06:54.629685 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-faa387f4-0845-4e16-8907-02cc70b4b938 admin admin] The token cache is being invalidate because project fdf48ef9191942658cb9984b3b86415d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:06:54.633518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-faa387f4-0845-4e16-8907-02cc70b4b938 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:06:54.633518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-faa387f4-0845-4e16-8907-02cc70b4b938 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:06:54.633518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 756/1510] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:06:54 2022] DELETE /identity/v3/projects/fdf48ef9191942658cb9984b3b86415d => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:07:03.760295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:07:04.625808 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:08:32.770864 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59586e84-9759-4b9b-9296-d4387d8a94ed None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:32.771510 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59586e84-9759-4b9b-9296-d4387d8a94ed None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:32.771510 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59586e84-9759-4b9b-9296-d4387d8a94ed None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:32.772794 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 15:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:08:32.777784 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e8e5c-2358-46b4-8eb3-1bfad69852cd None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:32.778229 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e8e5c-2358-46b4-8eb3-1bfad69852cd None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:32.778388 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e8e5c-2358-46b4-8eb3-1bfad69852cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:32.796174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-e72e8e5c-2358-46b4-8eb3-1bfad69852cd None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:08:32.797381 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e72e8e5c-2358-46b4-8eb3-1bfad69852cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:08:32.857899 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 15:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:08:33.011534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59586e84-9759-4b9b-9296-d4387d8a94ed None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:08:33.013235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-59586e84-9759-4b9b-9296-d4387d8a94ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:08:33.122467 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59586e84-9759-4b9b-9296-d4387d8a94ed None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:08:33.128355 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79b3bb-5f8c-4b91-ad24-7d284552b249 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:08:33.129049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79b3bb-5f8c-4b91-ad24-7d284552b249 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:33.129332 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79b3bb-5f8c-4b91-ad24-7d284552b249 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:33.129605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79b3bb-5f8c-4b91-ad24-7d284552b249 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:33.138660 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79b3bb-5f8c-4b91-ad24-7d284552b249 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:08:33.140303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79b3bb-5f8c-4b91-ad24-7d284552b249 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:08:33.150449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 756/1513] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:08:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:08:33.995793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da2e35-43f6-469f-a7cc-cf05bfc9b509 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:33.996344 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da2e35-43f6-469f-a7cc-cf05bfc9b509 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:33.996344 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da2e35-43f6-469f-a7cc-cf05bfc9b509 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:33.997750 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 15:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:08:34.002687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b99b41-fd8c-409e-bd32-f4cb28c9944f None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:34.003149 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b99b41-fd8c-409e-bd32-f4cb28c9944f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:34.003385 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b99b41-fd8c-409e-bd32-f4cb28c9944f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:34.022505 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-87b99b41-fd8c-409e-bd32-f4cb28c9944f None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:08:34.023737 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-87b99b41-fd8c-409e-bd32-f4cb28c9944f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:08:34.032021 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 15:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:08:34.185240 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da2e35-43f6-469f-a7cc-cf05bfc9b509 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:08:34.200516 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da2e35-43f6-469f-a7cc-cf05bfc9b509 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:08:34.205994 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8eff10-d164-4ccd-9877-1e29ebc7548d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:08:34.206733 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eff10-d164-4ccd-9877-1e29ebc7548d service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:34.207356 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eff10-d164-4ccd-9877-1e29ebc7548d service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:34.207476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eff10-d164-4ccd-9877-1e29ebc7548d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:34.216050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8eff10-d164-4ccd-9877-1e29ebc7548d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:08:34.217801 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8eff10-d164-4ccd-9877-1e29ebc7548d service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:08:34.228081 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 759/1516] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:08:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:08:34.757015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce53d79-c2ca-4765-b8ba-a9ca64b8d386 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:34.757319 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce53d79-c2ca-4765-b8ba-a9ca64b8d386 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:34.757596 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce53d79-c2ca-4765-b8ba-a9ca64b8d386 None None] PATH_INFO: `/v3` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:34.758947 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 758/1517] 10.222.0.33 () {54 vars in 925 bytes} [Sat Jul 16 15:08:34 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:08:34.765055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b668c-5202-466a-93e5-8df15429c478 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:34.765349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b668c-5202-466a-93e5-8df15429c478 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:34.765636 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b668c-5202-466a-93e5-8df15429c478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:34.814798 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-858b668c-5202-466a-93e5-8df15429c478 None None] MFA Rules not processed for user `3a30cee766814c8685578819261705ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:08:34.816214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-858b668c-5202-466a-93e5-8df15429c478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:08:34.889450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 760/1518] 10.222.0.33 () {58 vars in 1015 bytes} [Sat Jul 16 15:08:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:08:38.855046 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ffd2f-5578-42b1-b0b8-2b2be9b33377 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:38.855598 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ffd2f-5578-42b1-b0b8-2b2be9b33377 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:38.855598 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ffd2f-5578-42b1-b0b8-2b2be9b33377 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:38.857005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 759/1519] 10.222.0.27 () {54 vars in 960 bytes} [Sat Jul 16 15:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:08:38.862493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c94115-d0bf-48ad-8de0-282945d6b11a None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:38.863547 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c94115-d0bf-48ad-8de0-282945d6b11a None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:38.863723 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c94115-d0bf-48ad-8de0-282945d6b11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:38.882844 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-86c94115-d0bf-48ad-8de0-282945d6b11a None None] MFA Rules not processed for user `3a30cee766814c8685578819261705ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:08:38.884096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-86c94115-d0bf-48ad-8de0-282945d6b11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:08:38.892899 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 761/1520] 10.222.0.27 () {58 vars in 1057 bytes} [Sat Jul 16 15:08:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:08:38.900905 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ffd2f-5578-42b1-b0b8-2b2be9b33377 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:08:38.916776 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ffd2f-5578-42b1-b0b8-2b2be9b33377 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:08:38.922133 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8015d-cfd3-4e3c-8b37-b801e7b408a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a30cee766814c8685578819261705ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:08:38.922776 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8015d-cfd3-4e3c-8b37-b801e7b408a0 service placement] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:38.923095 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8015d-cfd3-4e3c-8b37-b801e7b408a0 service placement] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:38.923372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8015d-cfd3-4e3c-8b37-b801e7b408a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:38.933211 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8015d-cfd3-4e3c-8b37-b801e7b408a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:08:38.934868 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8015d-cfd3-4e3c-8b37-b801e7b408a0 service placement] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:08:38.948035 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 760/1521] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:08:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:08:45.472289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdd1e2-3a4a-4145-b186-d9949ecba608 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:45.472864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdd1e2-3a4a-4145-b186-d9949ecba608 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:45.472864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdd1e2-3a4a-4145-b186-d9949ecba608 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:45.474719 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 15:08:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:08:45.479911 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460c054-c3e4-4f79-b81c-ca1a62e794c1 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:45.480349 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460c054-c3e4-4f79-b81c-ca1a62e794c1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:45.480463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460c054-c3e4-4f79-b81c-ca1a62e794c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:45.498837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-6460c054-c3e4-4f79-b81c-ca1a62e794c1 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:08:45.500068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6460c054-c3e4-4f79-b81c-ca1a62e794c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:08:45.508061 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 15:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:08:45.662060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abdd1e2-3a4a-4145-b186-d9949ecba608 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:08:45.676714 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abdd1e2-3a4a-4145-b186-d9949ecba608 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:08:45.682907 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d213df-6b98-46e7-9232-ea19c06d44f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:08:45.683556 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d213df-6b98-46e7-9232-ea19c06d44f5 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:45.683833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d213df-6b98-46e7-9232-ea19c06d44f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:45.684104 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d213df-6b98-46e7-9232-ea19c06d44f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:45.692314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d213df-6b98-46e7-9232-ea19c06d44f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:08:45.694224 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d213df-6b98-46e7-9232-ea19c06d44f5 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:08:45.703961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 763/1524] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:08:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:08:46.531444 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60717a-00cf-451d-9afb-3fdf52fc1c23 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:46.532021 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60717a-00cf-451d-9afb-3fdf52fc1c23 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:46.532021 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60717a-00cf-451d-9afb-3fdf52fc1c23 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:46.533357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Jul 16 15:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:08:46.538358 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b48f5d-8083-438c-b5c3-352814763d01 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:46.538939 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b48f5d-8083-438c-b5c3-352814763d01 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:46.538939 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b48f5d-8083-438c-b5c3-352814763d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:46.557282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-36b48f5d-8083-438c-b5c3-352814763d01 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:08:46.559434 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-36b48f5d-8083-438c-b5c3-352814763d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:08:46.566461 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Jul 16 15:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:08:46.722345 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60717a-00cf-451d-9afb-3fdf52fc1c23 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:08:46.737461 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60717a-00cf-451d-9afb-3fdf52fc1c23 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:08:46.743170 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fcba8-56f5-42a2-818f-e97a8b6baf16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:08:46.743807 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fcba8-56f5-42a2-818f-e97a8b6baf16 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:46.744091 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fcba8-56f5-42a2-818f-e97a8b6baf16 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:46.744372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fcba8-56f5-42a2-818f-e97a8b6baf16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:46.752524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fcba8-56f5-42a2-818f-e97a8b6baf16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:08:46.754106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fcba8-56f5-42a2-818f-e97a8b6baf16 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:08:46.763674 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 763/1527] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:08:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:08:50.621295 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5508255-f885-4ed5-9342-b17126c73117 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:50.621841 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5508255-f885-4ed5-9342-b17126c73117 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:50.621841 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5508255-f885-4ed5-9342-b17126c73117 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:50.623319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 15:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:08:50.635178 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0871211-63c8-4944-9a41-e1b894545892 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:50.635178 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0871211-63c8-4944-9a41-e1b894545892 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:50.635178 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0871211-63c8-4944-9a41-e1b894545892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:50.647990 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f0871211-63c8-4944-9a41-e1b894545892 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:08:50.649261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f0871211-63c8-4944-9a41-e1b894545892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:08:50.657367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 15:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:08:50.697020 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5508255-f885-4ed5-9342-b17126c73117 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:08:50.699175 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b5508255-f885-4ed5-9342-b17126c73117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:08:50.729578 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:08:50.759170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5508255-f885-4ed5-9342-b17126c73117 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:08:50.763397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa84fd-d34a-4056-854f-edaa50b67e9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:08:50.764039 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa84fd-d34a-4056-854f-edaa50b67e9b service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:50.764313 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa84fd-d34a-4056-854f-edaa50b67e9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:50.764584 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa84fd-d34a-4056-854f-edaa50b67e9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:50.773629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa84fd-d34a-4056-854f-edaa50b67e9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:08:50.775137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa84fd-d34a-4056-854f-edaa50b67e9b service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:08:50.798463 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 766/1530] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:08:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:08:52.835765 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23896361-503b-4b00-b186-71eff8c50009 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:52.836430 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23896361-503b-4b00-b186-71eff8c50009 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:52.836430 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23896361-503b-4b00-b186-71eff8c50009 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:52.837681 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 15:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:08:52.843673 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239f017-9e2f-4cfa-8434-3701bf7d40fc None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:52.844225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239f017-9e2f-4cfa-8434-3701bf7d40fc None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:52.844225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239f017-9e2f-4cfa-8434-3701bf7d40fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:52.863172 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-6239f017-9e2f-4cfa-8434-3701bf7d40fc None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:08:52.864410 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-6239f017-9e2f-4cfa-8434-3701bf7d40fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:08:52.873693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 15:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:08:52.909523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23896361-503b-4b00-b186-71eff8c50009 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:08:52.921349 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23896361-503b-4b00-b186-71eff8c50009 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:08:52.924988 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd26878-2a6a-4a1e-9739-d3bba20e7da0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:08:52.925629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd26878-2a6a-4a1e-9739-d3bba20e7da0 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:52.925911 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd26878-2a6a-4a1e-9739-d3bba20e7da0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:52.926186 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd26878-2a6a-4a1e-9739-d3bba20e7da0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:52.936578 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd26878-2a6a-4a1e-9739-d3bba20e7da0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:08:52.938026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd26878-2a6a-4a1e-9739-d3bba20e7da0 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:08:52.949117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 766/1533] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:08:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:08:56.559657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a2270-be37-4d86-912c-d86f986367ed None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:56.565683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a2270-be37-4d86-912c-d86f986367ed None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:56.565683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a2270-be37-4d86-912c-d86f986367ed None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:56.565683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 15:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:08:56.566789 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2951f66-ac24-4824-b780-00c04dc67a46 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:56.567224 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2951f66-ac24-4824-b780-00c04dc67a46 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:56.567375 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2951f66-ac24-4824-b780-00c04dc67a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:56.574829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=74 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:08:56.586989 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c2951f66-ac24-4824-b780-00c04dc67a46 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:08:56.588181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c2951f66-ac24-4824-b780-00c04dc67a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:08:56.596594 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 15:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:08:56.904301 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a2270-be37-4d86-912c-d86f986367ed None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:08:56.906016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c67a2270-be37-4d86-912c-d86f986367ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:08:56.964767 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a2270-be37-4d86-912c-d86f986367ed None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:08:56.968204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe84cad-0b94-42f2-9e0e-c7204c2c565a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:08:56.968836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe84cad-0b94-42f2-9e0e-c7204c2c565a service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:56.969112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe84cad-0b94-42f2-9e0e-c7204c2c565a service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:56.969382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe84cad-0b94-42f2-9e0e-c7204c2c565a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:56.979267 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe84cad-0b94-42f2-9e0e-c7204c2c565a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:08:56.980687 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe84cad-0b94-42f2-9e0e-c7204c2c565a service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:08:56.990178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 769/1536] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:08:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:08:59.064314 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5710e-d38f-4ce0-bbc9-ae1086cad575 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:59.064853 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5710e-d38f-4ce0-bbc9-ae1086cad575 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:59.064853 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5710e-d38f-4ce0-bbc9-ae1086cad575 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:59.066216 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 916 bytes} [Sat Jul 16 15:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:08:59.072752 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0c59f-baa7-485e-8be5-6d307a08d06c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:59.072752 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0c59f-baa7-485e-8be5-6d307a08d06c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:59.075959 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0c59f-baa7-485e-8be5-6d307a08d06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:59.093068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-abe0c59f-baa7-485e-8be5-6d307a08d06c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:08:59.094285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-abe0c59f-baa7-485e-8be5-6d307a08d06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:08:59.102300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Sat Jul 16 15:08:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:08:59.412970 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5710e-d38f-4ce0-bbc9-ae1086cad575 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:08:59.414662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bfb5710e-d38f-4ce0-bbc9-ae1086cad575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:08:59.429108 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5710e-d38f-4ce0-bbc9-ae1086cad575 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:08:59.432672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e9a5c-c85e-40bc-95d4-08d7cd38807b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:08:59.433321 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e9a5c-c85e-40bc-95d4-08d7cd38807b service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:08:59.433597 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e9a5c-c85e-40bc-95d4-08d7cd38807b service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:08:59.433869 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e9a5c-c85e-40bc-95d4-08d7cd38807b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:08:59.444387 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e9a5c-c85e-40bc-95d4-08d7cd38807b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:08:59.445855 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e9a5c-c85e-40bc-95d4-08d7cd38807b service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:08:59.456006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 769/1539] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:08:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:09:09.087070 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=47 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:09:09.451550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=8 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:29.746194 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca690a-3253-49da-8546-aaf95e3b5127 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:29.746864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca690a-3253-49da-8546-aaf95e3b5127 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:29.746864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ca690a-3253-49da-8546-aaf95e3b5127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:29.770177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-71ca690a-3253-49da-8546-aaf95e3b5127 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:29.772078 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-71ca690a-3253-49da-8546-aaf95e3b5127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:29.780732 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 771/1540] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:29.810304 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c12e2-eba0-4e41-84d0-227e9d11078c None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:29.810962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c12e2-eba0-4e41-84d0-227e9d11078c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:29.810962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c12e2-eba0-4e41-84d0-227e9d11078c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:29.829414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0eb9a-3a00-4f62-b94f-f8136c6b2a90 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:29.829643 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0eb9a-3a00-4f62-b94f-f8136c6b2a90 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:29.829901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0eb9a-3a00-4f62-b94f-f8136c6b2a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:29.855710 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-bb5c12e2-eba0-4e41-84d0-227e9d11078c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:29.856934 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bb5c12e2-eba0-4e41-84d0-227e9d11078c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:29.876578 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-f9c0eb9a-3a00-4f62-b94f-f8136c6b2a90 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:29.887727 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 770/1541] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:29.892766 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c12e2-eba0-4e41-84d0-227e9d11078c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:29.892897 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-f9c0eb9a-3a00-4f62-b94f-f8136c6b2a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:29.909083 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 772/1542] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:29.921277 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a16f4-c5a5-4cc7-b3d8-43096823ac6b None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:29.921605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c12e2-eba0-4e41-84d0-227e9d11078c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:29.921725 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a16f4-c5a5-4cc7-b3d8-43096823ac6b None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:29.921853 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a16f4-c5a5-4cc7-b3d8-43096823ac6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:29.936691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfbb6f9-c79c-4976-95f3-46f01fada163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:29.937380 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbb6f9-c79c-4976-95f3-46f01fada163 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:29.937671 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbb6f9-c79c-4976-95f3-46f01fada163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:29.937946 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbb6f9-c79c-4976-95f3-46f01fada163 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:29.938750 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfbb6f9-c79c-4976-95f3-46f01fada163 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:10:29.939102 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfbb6f9-c79c-4976-95f3-46f01fada163 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:29.944319 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: WARNING py.warnings [None req-edfbb6f9-c79c-4976-95f3-46f01fada163 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 15:10:29.944319 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: warnings.warn(msg) Jul 16 15:10:29.944319 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: Jul 16 15:10:29.944319 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfbb6f9-c79c-4976-95f3-46f01fada163 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:29.952548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-243a16f4-c5a5-4cc7-b3d8-43096823ac6b None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:29.954355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-243a16f4-c5a5-4cc7-b3d8-43096823ac6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:29.968545 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 771/1543] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:10:29.968871 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 773/1544] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:10:29.977020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfbb6f9-c79c-4976-95f3-46f01fada163 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:29.978264 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a16f4-c5a5-4cc7-b3d8-43096823ac6b None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:29.989443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a16f4-c5a5-4cc7-b3d8-43096823ac6b None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:29.995284 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f2f9b-98fc-4cc1-990d-f54f08f8fe5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:29.995965 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f2f9b-98fc-4cc1-990d-f54f08f8fe5a admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:29.996635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f2f9b-98fc-4cc1-990d-f54f08f8fe5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:29.996839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f2f9b-98fc-4cc1-990d-f54f08f8fe5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:29.997558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f2f9b-98fc-4cc1-990d-f54f08f8fe5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:29.998822 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: WARNING py.warnings [None req-555f2f9b-98fc-4cc1-990d-f54f08f8fe5a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 15:10:29.998822 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: warnings.warn(msg) Jul 16 15:10:29.998822 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: Jul 16 15:10:29.999302 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfbb6f9-c79c-4976-95f3-46f01fada163 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:30.000199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f2f9b-98fc-4cc1-990d-f54f08f8fe5a admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:30.015639 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715647f0-d406-4575-a2bc-91ebdae31224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:30.016314 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715647f0-d406-4575-a2bc-91ebdae31224 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.016748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715647f0-d406-4575-a2bc-91ebdae31224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.017034 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715647f0-d406-4575-a2bc-91ebdae31224 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.018070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715647f0-d406-4575-a2bc-91ebdae31224 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:10:30.018381 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715647f0-d406-4575-a2bc-91ebdae31224 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:30.020038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715647f0-d406-4575-a2bc-91ebdae31224 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:30.044282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 772/1545] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:10:30.054896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715647f0-d406-4575-a2bc-91ebdae31224 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:30.068797 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715647f0-d406-4575-a2bc-91ebdae31224 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:30.079613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851eca65-28e6-42b0-94a6-d6658f375220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:30.080295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851eca65-28e6-42b0-94a6-d6658f375220 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.080580 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851eca65-28e6-42b0-94a6-d6658f375220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.080855 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851eca65-28e6-42b0-94a6-d6658f375220 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.081946 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851eca65-28e6-42b0-94a6-d6658f375220 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:30.084087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851eca65-28e6-42b0-94a6-d6658f375220 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:30.091211 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 774/1546] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:10:29 2022] POST /identity/v3/projects => generated 405 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:10:30.093164 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db41dc1-b9e7-4b01-95bd-46b2169628e2 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.100233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db41dc1-b9e7-4b01-95bd-46b2169628e2 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.101037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db41dc1-b9e7-4b01-95bd-46b2169628e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.128291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-8db41dc1-b9e7-4b01-95bd-46b2169628e2 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:30.129519 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-8db41dc1-b9e7-4b01-95bd-46b2169628e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:30.189337 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 773/1547] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:10:30 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:10:30.191811 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503ff51-c719-459e-9d4e-93bb1c10d5fe None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.192338 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503ff51-c719-459e-9d4e-93bb1c10d5fe None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.192936 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503ff51-c719-459e-9d4e-93bb1c10d5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.207158 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 775/1548] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:30.208189 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db41dc1-b9e7-4b01-95bd-46b2169628e2 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:30.217547 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-1503ff51-c719-459e-9d4e-93bb1c10d5fe None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:30.219105 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1503ff51-c719-459e-9d4e-93bb1c10d5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:30.219976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db41dc1-b9e7-4b01-95bd-46b2169628e2 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:30.282824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf2aff-a344-48c1-8000-facafba4675a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:30.283456 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf2aff-a344-48c1-8000-facafba4675a admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.283738 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf2aff-a344-48c1-8000-facafba4675a admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.284015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf2aff-a344-48c1-8000-facafba4675a admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.284490 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 774/1549] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:30.285790 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da500a6-cd3d-4ab5-a1bb-90767c2bb7f1 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.286076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da500a6-cd3d-4ab5-a1bb-90767c2bb7f1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.286920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da500a6-cd3d-4ab5-a1bb-90767c2bb7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.291234 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf2aff-a344-48c1-8000-facafba4675a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:30.293735 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf2aff-a344-48c1-8000-facafba4675a admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:30.306414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-1da500a6-cd3d-4ab5-a1bb-90767c2bb7f1 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:30.307684 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1da500a6-cd3d-4ab5-a1bb-90767c2bb7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:30.315845 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 775/1550] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:30.317558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ffad7-e1cf-4182-87f8-4b3f985c6a70 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.317894 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ffad7-e1cf-4182-87f8-4b3f985c6a70 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.318171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ffad7-e1cf-4182-87f8-4b3f985c6a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.336711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b86ffad7-e1cf-4182-87f8-4b3f985c6a70 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:30.338097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b86ffad7-e1cf-4182-87f8-4b3f985c6a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:30.346175 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 776/1551] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:30.349333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44991787-e140-4a74-9344-1b81526209ed None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.349333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44991787-e140-4a74-9344-1b81526209ed None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.349333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44991787-e140-4a74-9344-1b81526209ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.367507 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-44991787-e140-4a74-9344-1b81526209ed None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:30.368759 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-44991787-e140-4a74-9344-1b81526209ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:30.377039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 777/1552] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:30.378268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e4453-8727-4208-af94-75783201a1f2 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.381544 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e4453-8727-4208-af94-75783201a1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.381544 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e4453-8727-4208-af94-75783201a1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.398172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-279e4453-8727-4208-af94-75783201a1f2 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:30.399422 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-279e4453-8727-4208-af94-75783201a1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:30.404416 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 776/1553] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:10:30 2022] POST /identity/v3/users => generated 303 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:30.405294 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf2aff-a344-48c1-8000-facafba4675a admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:30.409358 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 778/1554] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:30.410667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc8636-223b-4286-ae78-e93a1940dcf5 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.410989 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc8636-223b-4286-ae78-e93a1940dcf5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.411267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc8636-223b-4286-ae78-e93a1940dcf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.416662 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf2aff-a344-48c1-8000-facafba4675a admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:30.422579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e35a2-76e3-46ef-8829-acffe6837340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:30.423278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e35a2-76e3-46ef-8829-acffe6837340 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.423560 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e35a2-76e3-46ef-8829-acffe6837340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.423830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e35a2-76e3-46ef-8829-acffe6837340 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.424523 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e35a2-76e3-46ef-8829-acffe6837340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:30.427095 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e35a2-76e3-46ef-8829-acffe6837340 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:30.430998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-74fc8636-223b-4286-ae78-e93a1940dcf5 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:30.432196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-74fc8636-223b-4286-ae78-e93a1940dcf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:30.440675 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 779/1555] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:30.441922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7463420-b7d5-4369-95a9-02fca56368df None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.442208 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7463420-b7d5-4369-95a9-02fca56368df None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.442485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7463420-b7d5-4369-95a9-02fca56368df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.460864 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a7463420-b7d5-4369-95a9-02fca56368df None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:30.462051 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a7463420-b7d5-4369-95a9-02fca56368df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:30.469964 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 780/1556] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:30.471925 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685767a-bb5c-4a1d-bb6a-15dd62ff102b None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.471925 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685767a-bb5c-4a1d-bb6a-15dd62ff102b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.471925 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685767a-bb5c-4a1d-bb6a-15dd62ff102b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.490530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a685767a-bb5c-4a1d-bb6a-15dd62ff102b None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:30.491770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a685767a-bb5c-4a1d-bb6a-15dd62ff102b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:30.499813 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 781/1557] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:30.501029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de41eb-8bdf-47a6-a428-a1dbfec600fe None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.501312 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de41eb-8bdf-47a6-a428-a1dbfec600fe None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.501590 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de41eb-8bdf-47a6-a428-a1dbfec600fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.516555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 777/1558] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:10:30 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:30.516555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0511c-ad77-441b-9f14-ce5d16a151a2 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.516555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0511c-ad77-441b-9f14-ce5d16a151a2 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.516555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0511c-ad77-441b-9f14-ce5d16a151a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.520980 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-09de41eb-8bdf-47a6-a428-a1dbfec600fe None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:30.522193 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-09de41eb-8bdf-47a6-a428-a1dbfec600fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:30.530266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 782/1559] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:30.531112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de41eb-8bdf-47a6-a428-a1dbfec600fe None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:30.535176 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-50e0511c-ad77-441b-9f14-ce5d16a151a2 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:30.537058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-50e0511c-ad77-441b-9f14-ce5d16a151a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:30.541291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de41eb-8bdf-47a6-a428-a1dbfec600fe None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:30.549445 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4be26-bfa7-4f49-be80-300181a11493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:30.549637 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 778/1560] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:30.549751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4be26-bfa7-4f49-be80-300181a11493 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.549751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4be26-bfa7-4f49-be80-300181a11493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.549751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4be26-bfa7-4f49-be80-300181a11493 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.550225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91671ad7-fc73-405e-8a82-45a7d59033b0 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.550225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91671ad7-fc73-405e-8a82-45a7d59033b0 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.550225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91671ad7-fc73-405e-8a82-45a7d59033b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.550524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4be26-bfa7-4f49-be80-300181a11493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:30.550524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4be26-bfa7-4f49-be80-300181a11493 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:30.563422 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 783/1561] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:30.563422 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4be26-bfa7-4f49-be80-300181a11493 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:30.569970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-91671ad7-fc73-405e-8a82-45a7d59033b0 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:30.569970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-91671ad7-fc73-405e-8a82-45a7d59033b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:30.575649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4be26-bfa7-4f49-be80-300181a11493 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:30.578039 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 779/1562] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:30.578939 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91671ad7-fc73-405e-8a82-45a7d59033b0 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:30.587313 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91184f64-5361-49ce-83ca-2015fd55d404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:30.588017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91184f64-5361-49ce-83ca-2015fd55d404 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.588284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91184f64-5361-49ce-83ca-2015fd55d404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.588564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91184f64-5361-49ce-83ca-2015fd55d404 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.589297 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91184f64-5361-49ce-83ca-2015fd55d404 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:10:30.589604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91184f64-5361-49ce-83ca-2015fd55d404 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:30.591330 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91184f64-5361-49ce-83ca-2015fd55d404 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:30.596368 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91671ad7-fc73-405e-8a82-45a7d59033b0 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:30.596368 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37aba0c-0867-494f-888b-c43c1a2e1267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:30.596995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37aba0c-0867-494f-888b-c43c1a2e1267 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.597282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37aba0c-0867-494f-888b-c43c1a2e1267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.597557 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37aba0c-0867-494f-888b-c43c1a2e1267 admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.598289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37aba0c-0867-494f-888b-c43c1a2e1267 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:10:30.598595 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37aba0c-0867-494f-888b-c43c1a2e1267 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:30.600947 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37aba0c-0867-494f-888b-c43c1a2e1267 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:30.604318 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 784/1563] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:10:30.605211 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91184f64-5361-49ce-83ca-2015fd55d404 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:30.615639 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91184f64-5361-49ce-83ca-2015fd55d404 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:30.618676 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 780/1564] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:10:30.619554 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37aba0c-0867-494f-888b-c43c1a2e1267 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:30.621083 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8020a-3b81-465b-8da2-ec8639bead22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:30.621752 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8020a-3b81-465b-8da2-ec8639bead22 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.622037 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8020a-3b81-465b-8da2-ec8639bead22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.622315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8020a-3b81-465b-8da2-ec8639bead22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.623079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8020a-3b81-465b-8da2-ec8639bead22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:10:30.623387 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8020a-3b81-465b-8da2-ec8639bead22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:30.625038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8020a-3b81-465b-8da2-ec8639bead22 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:30.628247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:30.632871 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37aba0c-0867-494f-888b-c43c1a2e1267 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:30.639337 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 785/1565] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:10:30.648946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb3d9df-06ca-434b-a2a2-e91735aeed56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:30.648946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3d9df-06ca-434b-a2a2-e91735aeed56 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.649195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8020a-3b81-465b-8da2-ec8639bead22 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:30.649294 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3d9df-06ca-434b-a2a2-e91735aeed56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.649294 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3d9df-06ca-434b-a2a2-e91735aeed56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.649294 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb3d9df-06ca-434b-a2a2-e91735aeed56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:30.649294 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb3d9df-06ca-434b-a2a2-e91735aeed56 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:30.651043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8020a-3b81-465b-8da2-ec8639bead22 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:30.656551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 781/1566] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:30.656551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb3d9df-06ca-434b-a2a2-e91735aeed56 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:30.658462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b906b96-f6e7-4494-8cb8-fd39456725f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:30.659151 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b906b96-f6e7-4494-8cb8-fd39456725f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.659498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b906b96-f6e7-4494-8cb8-fd39456725f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.659774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b906b96-f6e7-4494-8cb8-fd39456725f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.660502 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b906b96-f6e7-4494-8cb8-fd39456725f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:10:30.660809 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b906b96-f6e7-4494-8cb8-fd39456725f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:30.662581 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b906b96-f6e7-4494-8cb8-fd39456725f3 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:30.664736 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb3d9df-06ca-434b-a2a2-e91735aeed56 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:30.670610 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7144bea-9395-49bb-ac1e-fe292bc512e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:30.676658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7144bea-9395-49bb-ac1e-fe292bc512e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.676658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7144bea-9395-49bb-ac1e-fe292bc512e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.676658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7144bea-9395-49bb-ac1e-fe292bc512e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.676658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7144bea-9395-49bb-ac1e-fe292bc512e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:10:30.676658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7144bea-9395-49bb-ac1e-fe292bc512e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:30.676658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7144bea-9395-49bb-ac1e-fe292bc512e4 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:30.677797 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 786/1567] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:10:30.677797 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b906b96-f6e7-4494-8cb8-fd39456725f3 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:30.682166 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:30.687338 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b906b96-f6e7-4494-8cb8-fd39456725f3 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:30.691298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 782/1568] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:10:30.691988 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7144bea-9395-49bb-ac1e-fe292bc512e4 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:30.692151 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c75c3-df09-43c0-ad9f-01bb0884b17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:30.693441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c75c3-df09-43c0-ad9f-01bb0884b17a admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.693672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c75c3-df09-43c0-ad9f-01bb0884b17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.693922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c75c3-df09-43c0-ad9f-01bb0884b17a admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.694826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c75c3-df09-43c0-ad9f-01bb0884b17a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:30.696707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c75c3-df09-43c0-ad9f-01bb0884b17a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:30.704214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7144bea-9395-49bb-ac1e-fe292bc512e4 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:30.708513 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 787/1569] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:30.708513 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c75c3-df09-43c0-ad9f-01bb0884b17a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:30.712259 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c076d-b4ad-440c-8e68-2ba8adc62900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:30.712259 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c076d-b4ad-440c-8e68-2ba8adc62900 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.712259 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c076d-b4ad-440c-8e68-2ba8adc62900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.712259 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c076d-b4ad-440c-8e68-2ba8adc62900 admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.712628 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c076d-b4ad-440c-8e68-2ba8adc62900 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:10:30.712628 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c076d-b4ad-440c-8e68-2ba8adc62900 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:30.714031 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c076d-b4ad-440c-8e68-2ba8adc62900 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:30.719012 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c75c3-df09-43c0-ad9f-01bb0884b17a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:30.724091 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c627502-6d8f-4af5-832b-a5582ffb51f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:30.724769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c627502-6d8f-4af5-832b-a5582ffb51f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.725167 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c627502-6d8f-4af5-832b-a5582ffb51f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.725449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c627502-6d8f-4af5-832b-a5582ffb51f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.726144 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c627502-6d8f-4af5-832b-a5582ffb51f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:30.727225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 783/1570] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:10:30.728197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c076d-b4ad-440c-8e68-2ba8adc62900 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:30.728323 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c627502-6d8f-4af5-832b-a5582ffb51f9 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:30.738127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c076d-b4ad-440c-8e68-2ba8adc62900 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:30.743880 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975abef9-78dd-411c-98ee-85593738e40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:30.744543 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975abef9-78dd-411c-98ee-85593738e40b admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.744974 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975abef9-78dd-411c-98ee-85593738e40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.745256 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975abef9-78dd-411c-98ee-85593738e40b admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.745961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975abef9-78dd-411c-98ee-85593738e40b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:30.747924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975abef9-78dd-411c-98ee-85593738e40b admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:30.840090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 788/1571] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:10:30 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:30.840897 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c627502-6d8f-4af5-832b-a5582ffb51f9 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:30.860212 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 784/1572] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:10:30 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:30.860212 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975abef9-78dd-411c-98ee-85593738e40b admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:30.910765 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c627502-6d8f-4af5-832b-a5582ffb51f9 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:30.933688 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975abef9-78dd-411c-98ee-85593738e40b admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:30.938475 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b49dbe-b539-4d00-8d50-36c765d4017d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:30.939180 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b49dbe-b539-4d00-8d50-36c765d4017d admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.939477 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b49dbe-b539-4d00-8d50-36c765d4017d admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.939795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b49dbe-b539-4d00-8d50-36c765d4017d admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.940533 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b49dbe-b539-4d00-8d50-36c765d4017d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:30.942149 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b49dbe-b539-4d00-8d50-36c765d4017d admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:30.953653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 785/1573] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:30.955801 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b49dbe-b539-4d00-8d50-36c765d4017d admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:30.965705 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b49dbe-b539-4d00-8d50-36c765d4017d admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:30.970381 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ff302-74a7-4304-82cf-13c8a69963cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:30.971058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ff302-74a7-4304-82cf-13c8a69963cc admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.971339 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ff302-74a7-4304-82cf-13c8a69963cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.971608 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ff302-74a7-4304-82cf-13c8a69963cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.972305 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ff302-74a7-4304-82cf-13c8a69963cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:30.974336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57df856-2cb5-457e-8a73-067ee1494c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:30.974725 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ff302-74a7-4304-82cf-13c8a69963cc admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:30.975171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57df856-2cb5-457e-8a73-067ee1494c14 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:30.975562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57df856-2cb5-457e-8a73-067ee1494c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:30.975940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57df856-2cb5-457e-8a73-067ee1494c14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:30.976763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57df856-2cb5-457e-8a73-067ee1494c14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:30.978977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57df856-2cb5-457e-8a73-067ee1494c14 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:31.080149 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 786/1574] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:10:30 2022] POST /identity/v3/projects => generated 389 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:31.080976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ff302-74a7-4304-82cf-13c8a69963cc admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:31.125554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 789/1575] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:10:30 2022] POST /identity/v3/projects => generated 373 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:31.127041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57df856-2cb5-457e-8a73-067ee1494c14 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:31.147928 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ff302-74a7-4304-82cf-13c8a69963cc admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:31.173845 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:31.191739 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57df856-2cb5-457e-8a73-067ee1494c14 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:31.197159 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e38ca-7d74-4d2b-93b8-a02566793e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:31.198058 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e38ca-7d74-4d2b-93b8-a02566793e55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:31.198342 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e38ca-7d74-4d2b-93b8-a02566793e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:31.198619 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e38ca-7d74-4d2b-93b8-a02566793e55 admin admin] PATH_INFO: `/v3/projects/c7cd6660a7e34644944b849247efb37c/users/d8f7fb3933b3465bab5dd3f3a11e0317/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:31.201940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:31.209456 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98795c-8a77-4e4e-99ad-5aef01524b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:31.210258 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98795c-8a77-4e4e-99ad-5aef01524b9a admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:31.210545 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98795c-8a77-4e4e-99ad-5aef01524b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:31.210824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98795c-8a77-4e4e-99ad-5aef01524b9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:31.211587 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98795c-8a77-4e4e-99ad-5aef01524b9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:31.213638 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc98795c-8a77-4e4e-99ad-5aef01524b9a admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:31.214373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e38ca-7d74-4d2b-93b8-a02566793e55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7cd6660a7e34644944b849247efb37c, user_id=d8f7fb3933b3465bab5dd3f3a11e0317, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:31.217034 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e38ca-7d74-4d2b-93b8-a02566793e55 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:31.324284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 790/1576] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:10:31 2022] PUT /identity/v3/projects/c7cd6660a7e34644944b849247efb37c/users/d8f7fb3933b3465bab5dd3f3a11e0317/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:31.325145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e38ca-7d74-4d2b-93b8-a02566793e55 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:31.345534 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 787/1577] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:10:31 2022] POST /identity/v3/projects => generated 391 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:31.345534 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98795c-8a77-4e4e-99ad-5aef01524b9a admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:31.392756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e38ca-7d74-4d2b-93b8-a02566793e55 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:31.406532 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98795c-8a77-4e4e-99ad-5aef01524b9a admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:31.413439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4549830-b5e8-4dbf-ae5e-e0cce6bb57a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:31.414107 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4549830-b5e8-4dbf-ae5e-e0cce6bb57a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:31.414386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4549830-b5e8-4dbf-ae5e-e0cce6bb57a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:31.414659 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4549830-b5e8-4dbf-ae5e-e0cce6bb57a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:31.420517 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4549830-b5e8-4dbf-ae5e-e0cce6bb57a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:31.420517 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4549830-b5e8-4dbf-ae5e-e0cce6bb57a1 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:31.450751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2118cd-38c2-4bce-954a-30bac1b612df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:31.451552 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2118cd-38c2-4bce-954a-30bac1b612df admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:31.451833 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2118cd-38c2-4bce-954a-30bac1b612df admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:31.452107 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2118cd-38c2-4bce-954a-30bac1b612df admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:31.452825 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2118cd-38c2-4bce-954a-30bac1b612df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:31.454788 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2118cd-38c2-4bce-954a-30bac1b612df admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:31.513201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 788/1578] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:10:31 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:31.514084 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4549830-b5e8-4dbf-ae5e-e0cce6bb57a1 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:31.524468 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4549830-b5e8-4dbf-ae5e-e0cce6bb57a1 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:31.529075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c987e85-50e0-4b2a-a34a-898c95e4ad13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:31.529724 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c987e85-50e0-4b2a-a34a-898c95e4ad13 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:31.530025 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c987e85-50e0-4b2a-a34a-898c95e4ad13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:31.530280 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c987e85-50e0-4b2a-a34a-898c95e4ad13 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:31.530999 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c987e85-50e0-4b2a-a34a-898c95e4ad13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:31.532899 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c987e85-50e0-4b2a-a34a-898c95e4ad13 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:31.548269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 791/1579] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:10:31 2022] POST /identity/v3/projects => generated 371 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:31.549689 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2118cd-38c2-4bce-954a-30bac1b612df admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:31.611801 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2118cd-38c2-4bce-954a-30bac1b612df admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:31.617323 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5769a91-d8e8-4890-ae96-9fe918193a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:31.618207 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5769a91-d8e8-4890-ae96-9fe918193a2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:31.618480 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5769a91-d8e8-4890-ae96-9fe918193a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:31.618774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5769a91-d8e8-4890-ae96-9fe918193a2a admin admin] PATH_INFO: `/v3/projects/f1efed993ab5428593567b31a0a6cbad/users/e56835fbb6d14b5a8319d341440a03cd/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:31.626632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 789/1580] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:10:31 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:31.632895 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c987e85-50e0-4b2a-a34a-898c95e4ad13 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:31.637511 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5769a91-d8e8-4890-ae96-9fe918193a2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1efed993ab5428593567b31a0a6cbad, user_id=e56835fbb6d14b5a8319d341440a03cd, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:31.638238 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c987e85-50e0-4b2a-a34a-898c95e4ad13 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:31.640240 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5769a91-d8e8-4890-ae96-9fe918193a2a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:31.643072 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac2a4b-8aef-44f6-af03-b44987f7b67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:31.643693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac2a4b-8aef-44f6-af03-b44987f7b67e admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:31.643949 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac2a4b-8aef-44f6-af03-b44987f7b67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:31.644221 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac2a4b-8aef-44f6-af03-b44987f7b67e admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:31.644905 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac2a4b-8aef-44f6-af03-b44987f7b67e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:31.648123 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac2a4b-8aef-44f6-af03-b44987f7b67e admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:31.725792 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:31.730570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 792/1581] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:10:31 2022] PUT /identity/v3/projects/f1efed993ab5428593567b31a0a6cbad/users/e56835fbb6d14b5a8319d341440a03cd/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:31.731388 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5769a91-d8e8-4890-ae96-9fe918193a2a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:31.748791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 790/1582] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:10:31 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:31.750315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97527c89-7650-4247-8323-d7867c757e98 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:31.750651 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97527c89-7650-4247-8323-d7867c757e98 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:31.751144 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97527c89-7650-4247-8323-d7867c757e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:31.771566 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-97527c89-7650-4247-8323-d7867c757e98 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:31.773086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-97527c89-7650-4247-8323-d7867c757e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:31.794879 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5769a91-d8e8-4890-ae96-9fe918193a2a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:31.799627 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1eba4-ccb1-4280-a12d-51c9a446a7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:31.801009 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1eba4-ccb1-4280-a12d-51c9a446a7ad admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:31.801009 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1eba4-ccb1-4280-a12d-51c9a446a7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:31.801009 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1eba4-ccb1-4280-a12d-51c9a446a7ad admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:31.801598 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1eba4-ccb1-4280-a12d-51c9a446a7ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:31.802699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: WARNING py.warnings [None req-f4d1eba4-ccb1-4280-a12d-51c9a446a7ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 15:10:31.802699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: warnings.warn(msg) Jul 16 15:10:31.802699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: Jul 16 15:10:31.803926 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1eba4-ccb1-4280-a12d-51c9a446a7ad admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:31.835925 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 791/1583] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:31.836759 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97527c89-7650-4247-8323-d7867c757e98 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:31.846160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97527c89-7650-4247-8323-d7867c757e98 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:31.850952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe7d80-a0af-4980-82b3-c995c802c9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:31.851620 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe7d80-a0af-4980-82b3-c995c802c9a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:31.851904 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe7d80-a0af-4980-82b3-c995c802c9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:31.852181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe7d80-a0af-4980-82b3-c995c802c9a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:31.852882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe7d80-a0af-4980-82b3-c995c802c9a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:31.854806 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe7d80-a0af-4980-82b3-c995c802c9a4 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:31.861573 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:31.913052 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 793/1584] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:10:31 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:31.913884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1eba4-ccb1-4280-a12d-51c9a446a7ad admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:31.923675 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1eba4-ccb1-4280-a12d-51c9a446a7ad admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:31.928442 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea1051-1549-4e1f-a852-b1d0460a3994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:31.929087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea1051-1549-4e1f-a852-b1d0460a3994 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:31.929366 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea1051-1549-4e1f-a852-b1d0460a3994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:31.929638 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea1051-1549-4e1f-a852-b1d0460a3994 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:31.930350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea1051-1549-4e1f-a852-b1d0460a3994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:31.931996 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea1051-1549-4e1f-a852-b1d0460a3994 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:31.942271 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 794/1585] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:31 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:31.943459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea1051-1549-4e1f-a852-b1d0460a3994 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:31.950431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 792/1586] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:10:31 2022] POST /identity/v3/users => generated 296 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:31.951307 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe7d80-a0af-4980-82b3-c995c802c9a4 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:31.953637 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea1051-1549-4e1f-a852-b1d0460a3994 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:31.959062 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a271ff-6766-4f44-b025-7d067168f510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:31.959714 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a271ff-6766-4f44-b025-7d067168f510 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:31.959990 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a271ff-6766-4f44-b025-7d067168f510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:31.960259 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a271ff-6766-4f44-b025-7d067168f510 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:31.960943 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a271ff-6766-4f44-b025-7d067168f510 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:31.961713 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe7d80-a0af-4980-82b3-c995c802c9a4 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:31.962969 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a271ff-6766-4f44-b025-7d067168f510 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:31.966324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9497a3-5293-4ba1-aeb9-18be3a15807f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:31.967008 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9497a3-5293-4ba1-aeb9-18be3a15807f admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:31.967288 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9497a3-5293-4ba1-aeb9-18be3a15807f admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:31.967560 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9497a3-5293-4ba1-aeb9-18be3a15807f admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:31.968287 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9497a3-5293-4ba1-aeb9-18be3a15807f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:31.969857 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9497a3-5293-4ba1-aeb9-18be3a15807f admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:31.980335 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 793/1587] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:31.981581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd232f-ee0e-495d-abd4-b159ff1c31ff None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:31.981867 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd232f-ee0e-495d-abd4-b159ff1c31ff None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:31.982145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd232f-ee0e-495d-abd4-b159ff1c31ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.008117 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-40cd232f-ee0e-495d-abd4-b159ff1c31ff None None] MFA Rules not processed for user `e56835fbb6d14b5a8319d341440a03cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:32.010033 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-40cd232f-ee0e-495d-abd4-b159ff1c31ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:32.044497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 795/1588] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:10:31 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:32.047943 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a271ff-6766-4f44-b025-7d067168f510 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:32.056155 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a271ff-6766-4f44-b025-7d067168f510 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:32.061160 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a7aef-b581-450e-a416-c3f7e83a5106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:32.061827 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a7aef-b581-450e-a416-c3f7e83a5106 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.062108 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a7aef-b581-450e-a416-c3f7e83a5106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.062407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a7aef-b581-450e-a416-c3f7e83a5106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.063158 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a7aef-b581-450e-a416-c3f7e83a5106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:32.064890 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a7aef-b581-450e-a416-c3f7e83a5106 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:32.076808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 796/1589] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:32.077532 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 794/1590] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:32.077654 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a7aef-b581-450e-a416-c3f7e83a5106 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:32.078570 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd232f-ee0e-495d-abd4-b159ff1c31ff None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:32.091486 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd232f-ee0e-495d-abd4-b159ff1c31ff None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:32.096348 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e251c4-2029-4bc1-b111-7569a8c8da27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:32.097473 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e251c4-2029-4bc1-b111-7569a8c8da27 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.098147 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e251c4-2029-4bc1-b111-7569a8c8da27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.098246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e251c4-2029-4bc1-b111-7569a8c8da27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.099327 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e251c4-2029-4bc1-b111-7569a8c8da27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:32.105840 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e251c4-2029-4bc1-b111-7569a8c8da27 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:32.119871 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 795/1591] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:32.120881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e251c4-2029-4bc1-b111-7569a8c8da27 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:32.131383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e251c4-2029-4bc1-b111-7569a8c8da27 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:32.135972 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c828de-1b9c-4112-8de1-b206adefd6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:32.136629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c828de-1b9c-4112-8de1-b206adefd6fe admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.136911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c828de-1b9c-4112-8de1-b206adefd6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.137186 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c828de-1b9c-4112-8de1-b206adefd6fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.137909 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c828de-1b9c-4112-8de1-b206adefd6fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:32.139542 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c828de-1b9c-4112-8de1-b206adefd6fe admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:32.145557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a7aef-b581-450e-a416-c3f7e83a5106 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:32.150135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133e9ba-2a1d-4c1e-8d41-00423c71adcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:32.157551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 796/1592] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:32.157701 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133e9ba-2a1d-4c1e-8d41-00423c71adcb service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.157799 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c828de-1b9c-4112-8de1-b206adefd6fe admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:32.157915 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133e9ba-2a1d-4c1e-8d41-00423c71adcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.157915 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133e9ba-2a1d-4c1e-8d41-00423c71adcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.162212 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133e9ba-2a1d-4c1e-8d41-00423c71adcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:32.162797 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c828de-1b9c-4112-8de1-b206adefd6fe admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:32.163703 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133e9ba-2a1d-4c1e-8d41-00423c71adcb service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:32.167536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83849c-0ad9-44f4-9a80-96c9224391a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:32.168186 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83849c-0ad9-44f4-9a80-96c9224391a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.168449 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83849c-0ad9-44f4-9a80-96c9224391a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.168732 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83849c-0ad9-44f4-9a80-96c9224391a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.169516 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83849c-0ad9-44f4-9a80-96c9224391a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:32.171171 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83849c-0ad9-44f4-9a80-96c9224391a4 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:32.174073 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 797/1593] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:10:32.174912 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133e9ba-2a1d-4c1e-8d41-00423c71adcb service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:32.183053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 797/1594] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:32.188005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83849c-0ad9-44f4-9a80-96c9224391a4 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:32.188125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133e9ba-2a1d-4c1e-8d41-00423c71adcb service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:32.192292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505cff47-2641-4da5-aded-c5cca45aa87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:32.193589 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cff47-2641-4da5-aded-c5cca45aa87d admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.193870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cff47-2641-4da5-aded-c5cca45aa87d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.194144 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cff47-2641-4da5-aded-c5cca45aa87d admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.194864 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505cff47-2641-4da5-aded-c5cca45aa87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:32.195223 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83849c-0ad9-44f4-9a80-96c9224391a4 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:32.197362 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505cff47-2641-4da5-aded-c5cca45aa87d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:32.200512 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5347529-bb66-4423-9fbb-cdf35f7f82d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:32.201182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5347529-bb66-4423-9fbb-cdf35f7f82d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.201519 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5347529-bb66-4423-9fbb-cdf35f7f82d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.201824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5347529-bb66-4423-9fbb-cdf35f7f82d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.202514 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5347529-bb66-4423-9fbb-cdf35f7f82d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:32.204181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5347529-bb66-4423-9fbb-cdf35f7f82d0 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:32.213599 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 798/1595] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:32.213599 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505cff47-2641-4da5-aded-c5cca45aa87d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:32.220861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 798/1596] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:32.220861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc4889-fd80-41e8-8f30-76b2ae4a15ff None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.220861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc4889-fd80-41e8-8f30-76b2ae4a15ff None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.220861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc4889-fd80-41e8-8f30-76b2ae4a15ff None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.220861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 799/1597] 10.222.0.27 () {54 vars in 961 bytes} [Sat Jul 16 15:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:10:32.220861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc4889-fd80-41e8-8f30-76b2ae4a15ff None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:32.227777 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505cff47-2641-4da5-aded-c5cca45aa87d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:32.232896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316a7b6-1775-46b8-8679-18ab94fea5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:32.233559 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316a7b6-1775-46b8-8679-18ab94fea5e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.233730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc4889-fd80-41e8-8f30-76b2ae4a15ff None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:32.233845 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316a7b6-1775-46b8-8679-18ab94fea5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.234096 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316a7b6-1775-46b8-8679-18ab94fea5e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.234847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316a7b6-1775-46b8-8679-18ab94fea5e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:32.236550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316a7b6-1775-46b8-8679-18ab94fea5e7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:32.238314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd55d02-7520-477c-9b49-8b81b5db87ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:32.239001 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd55d02-7520-477c-9b49-8b81b5db87ea admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.239284 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd55d02-7520-477c-9b49-8b81b5db87ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.240087 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd55d02-7520-477c-9b49-8b81b5db87ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.241841 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd55d02-7520-477c-9b49-8b81b5db87ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:32.243470 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd55d02-7520-477c-9b49-8b81b5db87ea admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:32.248721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 799/1598] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:32.252904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316a7b6-1775-46b8-8679-18ab94fea5e7 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:32.261952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 800/1599] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:32.261952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd55d02-7520-477c-9b49-8b81b5db87ea admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:32.262235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316a7b6-1775-46b8-8679-18ab94fea5e7 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:32.264887 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7bfeb-92b9-45f8-86bb-621f86db410e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:32.265787 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7bfeb-92b9-45f8-86bb-621f86db410e admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.266068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7bfeb-92b9-45f8-86bb-621f86db410e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.266343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7bfeb-92b9-45f8-86bb-621f86db410e admin admin] PATH_INFO: `/v3/projects/909b9d1ab80d456b8801692973d6799d/users/19104226c2a64fcab522a5f2ade6fc3a/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.278067 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd55d02-7520-477c-9b49-8b81b5db87ea admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:32.278067 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e9dea-e15c-451f-aef2-89d857a3d749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:32.278635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e9dea-e15c-451f-aef2-89d857a3d749 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.278943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e9dea-e15c-451f-aef2-89d857a3d749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.279217 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e9dea-e15c-451f-aef2-89d857a3d749 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.279998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e9dea-e15c-451f-aef2-89d857a3d749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:32.281834 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e9dea-e15c-451f-aef2-89d857a3d749 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:32.283334 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7bfeb-92b9-45f8-86bb-621f86db410e admin admin] RBAC: Authorizing `identity:create_grant(project_id=909b9d1ab80d456b8801692973d6799d, user_id=19104226c2a64fcab522a5f2ade6fc3a, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:32.285903 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7bfeb-92b9-45f8-86bb-621f86db410e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:32.294360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 801/1600] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:32.299263 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e9dea-e15c-451f-aef2-89d857a3d749 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:32.306690 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e9dea-e15c-451f-aef2-89d857a3d749 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:32.312274 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b119831-b6c3-43fd-bb5e-1f473d1ea71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:32.313174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b119831-b6c3-43fd-bb5e-1f473d1ea71a admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.313449 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b119831-b6c3-43fd-bb5e-1f473d1ea71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.313720 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b119831-b6c3-43fd-bb5e-1f473d1ea71a admin admin] PATH_INFO: `/v3/projects/009b3494f3794d3e8011b85dfe2d2d9b/users/096fd1b2e2b4482da1026d3fb7e192c2/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.330484 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b119831-b6c3-43fd-bb5e-1f473d1ea71a admin admin] RBAC: Authorizing `identity:create_grant(project_id=009b3494f3794d3e8011b85dfe2d2d9b, user_id=096fd1b2e2b4482da1026d3fb7e192c2, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:32.331667 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: WARNING py.warnings [None req-0b119831-b6c3-43fd-bb5e-1f473d1ea71a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 15:10:32.331667 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: warnings.warn(msg) Jul 16 15:10:32.331667 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: Jul 16 15:10:32.333908 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b119831-b6c3-43fd-bb5e-1f473d1ea71a admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:32.381869 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 800/1601] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:10:32 2022] PUT /identity/v3/projects/909b9d1ab80d456b8801692973d6799d/users/19104226c2a64fcab522a5f2ade6fc3a/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:32.382742 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7bfeb-92b9-45f8-86bb-621f86db410e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:32.391337 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:32.413085 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 802/1602] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:10:32 2022] PUT /identity/v3/projects/009b3494f3794d3e8011b85dfe2d2d9b/users/096fd1b2e2b4482da1026d3fb7e192c2/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:32.413085 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2b377-8f4c-4652-bdae-26f6b10dc1a3 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.413085 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2b377-8f4c-4652-bdae-26f6b10dc1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.413085 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2b377-8f4c-4652-bdae-26f6b10dc1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.448994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7bfeb-92b9-45f8-86bb-621f86db410e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:32.451870 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-2af2b377-8f4c-4652-bdae-26f6b10dc1a3 None None] MFA Rules not processed for user `3f2e241494244edf8d283e8f44192c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:32.453090 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-2af2b377-8f4c-4652-bdae-26f6b10dc1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:32.473457 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:32.521341 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf17d9-625a-423f-b44a-d30a3dadda7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:32.522098 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf17d9-625a-423f-b44a-d30a3dadda7b admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.522494 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf17d9-625a-423f-b44a-d30a3dadda7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.522903 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf17d9-625a-423f-b44a-d30a3dadda7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.523768 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf17d9-625a-423f-b44a-d30a3dadda7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:32.527165 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf17d9-625a-423f-b44a-d30a3dadda7b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:32.539308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 801/1603] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:32.540187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf17d9-625a-423f-b44a-d30a3dadda7b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:32.544222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 803/1604] 10.222.0.27 () {58 vars in 1058 bytes} [Sat Jul 16 15:10:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:32.545241 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2b377-8f4c-4652-bdae-26f6b10dc1a3 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:32.552987 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf17d9-625a-423f-b44a-d30a3dadda7b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:32.558859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eccbcff-8e33-4e67-abaa-1419af7b3997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:32.560053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccbcff-8e33-4e67-abaa-1419af7b3997 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.560310 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2b377-8f4c-4652-bdae-26f6b10dc1a3 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:32.560469 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccbcff-8e33-4e67-abaa-1419af7b3997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.560907 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccbcff-8e33-4e67-abaa-1419af7b3997 admin admin] PATH_INFO: `/v3/projects/2f6efc6860644128973230d227ad005a/users/9ee0106af0894acb911831433ca22caf/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.571003 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6c0db-2adc-4a49-bc3f-f44d33793434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:32.571003 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6c0db-2adc-4a49-bc3f-f44d33793434 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.571003 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6c0db-2adc-4a49-bc3f-f44d33793434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.571003 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6c0db-2adc-4a49-bc3f-f44d33793434 admin admin] PATH_INFO: `/v3/projects/bbea395ccd654ee2af057e8ef3b5451c/users/d8110cff925b49b8b8b3a44cc1891fcf/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.582174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eccbcff-8e33-4e67-abaa-1419af7b3997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f6efc6860644128973230d227ad005a, user_id=9ee0106af0894acb911831433ca22caf, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:32.586529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6c0db-2adc-4a49-bc3f-f44d33793434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbea395ccd654ee2af057e8ef3b5451c, user_id=d8110cff925b49b8b8b3a44cc1891fcf, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:32.587277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eccbcff-8e33-4e67-abaa-1419af7b3997 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:32.589633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6c0db-2adc-4a49-bc3f-f44d33793434 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:32.670965 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 802/1606] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:10:32 2022] PUT /identity/v3/projects/2f6efc6860644128973230d227ad005a/users/9ee0106af0894acb911831433ca22caf/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:32.671176 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 804/1606] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:10:32 2022] PUT /identity/v3/projects/bbea395ccd654ee2af057e8ef3b5451c/users/d8110cff925b49b8b8b3a44cc1891fcf/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:32.671278 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eccbcff-8e33-4e67-abaa-1419af7b3997 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:32.671368 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3748b1-b853-47f5-918a-a08cf12551fe None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.671368 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3748b1-b853-47f5-918a-a08cf12551fe None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.671368 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3748b1-b853-47f5-918a-a08cf12551fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.689513 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-ec3748b1-b853-47f5-918a-a08cf12551fe None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:32.691101 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ec3748b1-b853-47f5-918a-a08cf12551fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:32.755876 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eccbcff-8e33-4e67-abaa-1419af7b3997 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:32.761663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6147c-6da6-4e99-9cdf-5646643d07c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:32.762985 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6147c-6da6-4e99-9cdf-5646643d07c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.763474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6147c-6da6-4e99-9cdf-5646643d07c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.763919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6147c-6da6-4e99-9cdf-5646643d07c5 admin admin] PATH_INFO: `/v3/projects/e7ad41a7c9214f44a07aba8b9f86c6f4/users/2517f9ee6c3b4d52b44d5da510e7cbf8/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.788246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 805/1607] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:32.790118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db52086-42da-4fd0-b822-0c452f316bfc None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.790599 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db52086-42da-4fd0-b822-0c452f316bfc None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.791079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db52086-42da-4fd0-b822-0c452f316bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.803205 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6147c-6da6-4e99-9cdf-5646643d07c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7ad41a7c9214f44a07aba8b9f86c6f4, user_id=2517f9ee6c3b4d52b44d5da510e7cbf8, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:32.806160 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6147c-6da6-4e99-9cdf-5646643d07c5 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:32.818337 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-8db52086-42da-4fd0-b822-0c452f316bfc None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:32.820513 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-8db52086-42da-4fd0-b822-0c452f316bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:32.830296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 806/1608] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:32.831320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db52086-42da-4fd0-b822-0c452f316bfc None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:32.844032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db52086-42da-4fd0-b822-0c452f316bfc None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:32.849774 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4e820-1826-4ec7-84bb-d1cd4f9cb07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:32.851060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4e820-1826-4ec7-84bb-d1cd4f9cb07a admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.851448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4e820-1826-4ec7-84bb-d1cd4f9cb07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.851825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4e820-1826-4ec7-84bb-d1cd4f9cb07a admin admin] PATH_INFO: `/v3/projects/16830c8c8eea4169bb6d7aeae2d051e1/users/d0ff150bf5e8404c90e1e9fbe85fffe5/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:32.871832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4e820-1826-4ec7-84bb-d1cd4f9cb07a admin admin] RBAC: Authorizing `identity:create_grant(project_id=16830c8c8eea4169bb6d7aeae2d051e1, user_id=d0ff150bf5e8404c90e1e9fbe85fffe5, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:32.875372 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4e820-1826-4ec7-84bb-d1cd4f9cb07a admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:32.927832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 803/1609] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:10:32 2022] PUT /identity/v3/projects/e7ad41a7c9214f44a07aba8b9f86c6f4/users/2517f9ee6c3b4d52b44d5da510e7cbf8/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:32.928447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6147c-6da6-4e99-9cdf-5646643d07c5 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:32.944584 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:32.995648 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 807/1610] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:10:32 2022] PUT /identity/v3/projects/16830c8c8eea4169bb6d7aeae2d051e1/users/d0ff150bf5e8404c90e1e9fbe85fffe5/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:32.997303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457eaa65-5bf5-41b3-9f9b-17109bdd2579 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:32.997687 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457eaa65-5bf5-41b3-9f9b-17109bdd2579 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:32.998075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457eaa65-5bf5-41b3-9f9b-17109bdd2579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:33.027652 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-457eaa65-5bf5-41b3-9f9b-17109bdd2579 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:33.028387 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6147c-6da6-4e99-9cdf-5646643d07c5 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:33.029378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-457eaa65-5bf5-41b3-9f9b-17109bdd2579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:33.109310 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 808/1611] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:33.110982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbd50c-ae37-497c-af14-3365b300235f None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:33.111276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbd50c-ae37-497c-af14-3365b300235f None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:33.111597 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbd50c-ae37-497c-af14-3365b300235f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:33.126932 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:33.145161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f8394-da2f-4d54-9368-bc428f0eab75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:33.146602 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f8394-da2f-4d54-9368-bc428f0eab75 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:33.147025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f8394-da2f-4d54-9368-bc428f0eab75 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:33.147793 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f8394-da2f-4d54-9368-bc428f0eab75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:33.150578 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-71fbd50c-ae37-497c-af14-3365b300235f None None] MFA Rules not processed for user `d8110cff925b49b8b8b3a44cc1891fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:33.153266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-71fbd50c-ae37-497c-af14-3365b300235f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:33.216354 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66f8394-da2f-4d54-9368-bc428f0eab75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:33.218050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66f8394-da2f-4d54-9368-bc428f0eab75 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:33.231839 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 804/1612] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:10:33.232883 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f8394-da2f-4d54-9368-bc428f0eab75 service nova] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:33.234435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e66f8394-da2f-4d54-9368-bc428f0eab75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:33.234576 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 809/1613] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:33.235304 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbd50c-ae37-497c-af14-3365b300235f None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:33.236781 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-71fbd50c-ae37-497c-af14-3365b300235f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:33.297312 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:33.309162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f8394-da2f-4d54-9368-bc428f0eab75 service nova] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:33.313186 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61a810-dddf-4bd6-acb5-daa9fc712d90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:33.313867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61a810-dddf-4bd6-acb5-daa9fc712d90 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:33.314144 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61a810-dddf-4bd6-acb5-daa9fc712d90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:33.314446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61a810-dddf-4bd6-acb5-daa9fc712d90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:33.321685 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbd50c-ae37-497c-af14-3365b300235f None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:33.324889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61a810-dddf-4bd6-acb5-daa9fc712d90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:33.326362 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61a810-dddf-4bd6-acb5-daa9fc712d90 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:33.326950 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c67b82-6364-4bff-bd5f-5fd54ecaad2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:33.327854 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c67b82-6364-4bff-bd5f-5fd54ecaad2d service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:33.328344 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c67b82-6364-4bff-bd5f-5fd54ecaad2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:33.328775 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c67b82-6364-4bff-bd5f-5fd54ecaad2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:33.344494 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c67b82-6364-4bff-bd5f-5fd54ecaad2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:33.346263 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c67b82-6364-4bff-bd5f-5fd54ecaad2d service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:33.356176 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 805/1614] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:10:33.357050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130743ec-231b-49c2-843f-5ef4968bf60b None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:33.357439 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130743ec-231b-49c2-843f-5ef4968bf60b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:33.357835 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130743ec-231b-49c2-843f-5ef4968bf60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:33.384013 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 810/1615] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:10:33.387607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8aa22-6e35-4e2c-a78d-48a2ce77fd8c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:33.388001 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8aa22-6e35-4e2c-a78d-48a2ce77fd8c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:33.396508 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8aa22-6e35-4e2c-a78d-48a2ce77fd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:33.405081 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-130743ec-231b-49c2-843f-5ef4968bf60b None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:33.410357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-130743ec-231b-49c2-843f-5ef4968bf60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:33.433212 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 806/1616] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:33.434340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130743ec-231b-49c2-843f-5ef4968bf60b None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:33.440576 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-1fa8aa22-6e35-4e2c-a78d-48a2ce77fd8c None None] MFA Rules not processed for user `d0ff150bf5e8404c90e1e9fbe85fffe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:33.441882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-1fa8aa22-6e35-4e2c-a78d-48a2ce77fd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:33.452553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130743ec-231b-49c2-843f-5ef4968bf60b None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:33.457259 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e590d857-16f2-4163-bb85-daa993eb324e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:33.457669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e590d857-16f2-4163-bb85-daa993eb324e service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:33.458055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e590d857-16f2-4163-bb85-daa993eb324e service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:33.458290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e590d857-16f2-4163-bb85-daa993eb324e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:33.468161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e590d857-16f2-4163-bb85-daa993eb324e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:33.469530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e590d857-16f2-4163-bb85-daa993eb324e service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:33.484040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 807/1617] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:10:33.488646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782aa24-9111-4f18-bafb-8e747789d362 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:33.488646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782aa24-9111-4f18-bafb-8e747789d362 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:33.488646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782aa24-9111-4f18-bafb-8e747789d362 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:33.492515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 808/1618] 10.222.0.27 () {54 vars in 961 bytes} [Sat Jul 16 15:10:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:10:33.493449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782aa24-9111-4f18-bafb-8e747789d362 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:33.504884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782aa24-9111-4f18-bafb-8e747789d362 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:33.521152 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b9042-68fc-4117-8f7b-9e0368e22104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:33.522082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b9042-68fc-4117-8f7b-9e0368e22104 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:33.522365 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b9042-68fc-4117-8f7b-9e0368e22104 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:33.522646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b9042-68fc-4117-8f7b-9e0368e22104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:33.535631 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b9042-68fc-4117-8f7b-9e0368e22104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:33.536904 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 811/1619] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:33.538082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057e6e4-c4e0-448c-aaca-39224715c24d None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:33.538380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057e6e4-c4e0-448c-aaca-39224715c24d None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:33.538722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057e6e4-c4e0-448c-aaca-39224715c24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:33.540000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b9042-68fc-4117-8f7b-9e0368e22104 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:33.555243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 809/1620] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:10:33.561092 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8f1a6-1b96-4223-ae4b-02574b9dce8d None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:33.561732 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8f1a6-1b96-4223-ae4b-02574b9dce8d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:33.562072 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8f1a6-1b96-4223-ae4b-02574b9dce8d None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:33.569342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-3057e6e4-c4e0-448c-aaca-39224715c24d None None] MFA Rules not processed for user `3f2e241494244edf8d283e8f44192c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:33.582351 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3057e6e4-c4e0-448c-aaca-39224715c24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:33.582534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 810/1621] 10.222.0.27 () {54 vars in 912 bytes} [Sat Jul 16 15:10:33 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:10:33.582534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8f1a6-1b96-4223-ae4b-02574b9dce8d None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:33.588604 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 812/1622] 10.222.0.27 () {58 vars in 1058 bytes} [Sat Jul 16 15:10:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:33.589907 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2ace6-ca42-4df1-adf7-58a15a6c2ed4 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:33.590550 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2ace6-ca42-4df1-adf7-58a15a6c2ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:33.590760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2ace6-ca42-4df1-adf7-58a15a6c2ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:33.606978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8f1a6-1b96-4223-ae4b-02574b9dce8d None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:33.616642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-31a2ace6-ca42-4df1-adf7-58a15a6c2ed4 None None] MFA Rules not processed for user `3f2e241494244edf8d283e8f44192c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:33.617153 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28e5f2-9022-412c-9f54-87ddfb48764d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:33.618022 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-31a2ace6-ca42-4df1-adf7-58a15a6c2ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:33.618845 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28e5f2-9022-412c-9f54-87ddfb48764d admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:33.619138 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28e5f2-9022-412c-9f54-87ddfb48764d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:33.620204 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28e5f2-9022-412c-9f54-87ddfb48764d admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:33.621304 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28e5f2-9022-412c-9f54-87ddfb48764d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:33.625370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28e5f2-9022-412c-9f54-87ddfb48764d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:33.638602 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 813/1623] 10.222.0.27 () {58 vars in 1009 bytes} [Sat Jul 16 15:10:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:33.644759 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2ace6-ca42-4df1-adf7-58a15a6c2ed4 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:33.660157 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2ace6-ca42-4df1-adf7-58a15a6c2ed4 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:33.668804 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41cae4-4406-47b6-b4c0-42972fd53b29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:33.670426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41cae4-4406-47b6-b4c0-42972fd53b29 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:33.670701 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41cae4-4406-47b6-b4c0-42972fd53b29 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:33.671023 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41cae4-4406-47b6-b4c0-42972fd53b29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:33.681058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f41cae4-4406-47b6-b4c0-42972fd53b29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:33.682825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f41cae4-4406-47b6-b4c0-42972fd53b29 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:33.687431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:33.694533 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 814/1624] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:10:33.695330 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41cae4-4406-47b6-b4c0-42972fd53b29 service nova] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:33.757417 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 811/1625] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:10:33 2022] POST /identity/v3/projects => generated 373 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:33.764554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28e5f2-9022-412c-9f54-87ddfb48764d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:33.789453 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41cae4-4406-47b6-b4c0-42972fd53b29 service nova] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:33.855764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66e40e-cb49-489f-b5eb-436a8e8982a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:33.856874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66e40e-cb49-489f-b5eb-436a8e8982a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:33.857160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66e40e-cb49-489f-b5eb-436a8e8982a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:33.857483 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66e40e-cb49-489f-b5eb-436a8e8982a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:34.017539 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28e5f2-9022-412c-9f54-87ddfb48764d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:34.038793 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d9d9e9-39cd-46d3-a662-6e8262b5ed35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:34.041255 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9d9e9-39cd-46d3-a662-6e8262b5ed35 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:34.041773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9d9e9-39cd-46d3-a662-6e8262b5ed35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:34.042271 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d9d9e9-39cd-46d3-a662-6e8262b5ed35 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:34.049765 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9d9e9-39cd-46d3-a662-6e8262b5ed35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:34.051742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66e40e-cb49-489f-b5eb-436a8e8982a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:34.056937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d9d9e9-39cd-46d3-a662-6e8262b5ed35 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:34.059280 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66e40e-cb49-489f-b5eb-436a8e8982a4 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:34.086238 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 815/1626] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:10:34.090051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b72f2-5307-4ba7-9117-d574a76ac48c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:34.090495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b72f2-5307-4ba7-9117-d574a76ac48c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:34.090784 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b72f2-5307-4ba7-9117-d574a76ac48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:34.132375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-1a5b72f2-5307-4ba7-9117-d574a76ac48c None None] MFA Rules not processed for user `d8f7fb3933b3465bab5dd3f3a11e0317`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:34.133797 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-1a5b72f2-5307-4ba7-9117-d574a76ac48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:34.187805 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:34.215968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 812/1627] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:10:33 2022] POST /identity/v3/users => generated 293 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:34.215968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1176dd-18b3-4ce4-880d-451a13cdf254 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:34.215968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1176dd-18b3-4ce4-880d-451a13cdf254 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:34.215968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1176dd-18b3-4ce4-880d-451a13cdf254 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:34.224350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 813/1628] 10.222.0.27 () {54 vars in 912 bytes} [Sat Jul 16 15:10:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:10:34.224350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1176dd-18b3-4ce4-880d-451a13cdf254 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:34.226451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 816/1629] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:34 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:34.230422 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08884b67-ee50-4930-a943-b168a2937407 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:34.231853 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08884b67-ee50-4930-a943-b168a2937407 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:34.232487 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08884b67-ee50-4930-a943-b168a2937407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:34.247192 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1176dd-18b3-4ce4-880d-451a13cdf254 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:34.256064 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276b6ad-b60c-4590-af91-629b2b4991c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:34.257159 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276b6ad-b60c-4590-af91-629b2b4991c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:34.257519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276b6ad-b60c-4590-af91-629b2b4991c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:34.257914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276b6ad-b60c-4590-af91-629b2b4991c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:34.258899 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276b6ad-b60c-4590-af91-629b2b4991c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:34.260800 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276b6ad-b60c-4590-af91-629b2b4991c6 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:34.279953 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-08884b67-ee50-4930-a943-b168a2937407 None None] MFA Rules not processed for user `3f2e241494244edf8d283e8f44192c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:34.281521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-08884b67-ee50-4930-a943-b168a2937407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:34.291345 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 814/1630] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:34.293854 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276b6ad-b60c-4590-af91-629b2b4991c6 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:34.309658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 817/1631] 10.222.0.27 () {58 vars in 1009 bytes} [Sat Jul 16 15:10:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:34.310865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08884b67-ee50-4930-a943-b168a2937407 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:34.311648 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276b6ad-b60c-4590-af91-629b2b4991c6 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:34.325257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08884b67-ee50-4930-a943-b168a2937407 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:34.331684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a863d6-cd13-422a-8696-db89cf36ab7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:34.332900 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a863d6-cd13-422a-8696-db89cf36ab7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:34.334139 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a863d6-cd13-422a-8696-db89cf36ab7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:34.334598 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a863d6-cd13-422a-8696-db89cf36ab7f admin admin] PATH_INFO: `/v3/projects/315f9034b39d43c98ef9064b7da5ff3e/users/64b055cfc9be4b4bb892f09eaba782ca/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:34.335458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb885b7-66a0-46ac-8923-6a20b613b764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:34.340154 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb885b7-66a0-46ac-8923-6a20b613b764 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:34.340154 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb885b7-66a0-46ac-8923-6a20b613b764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:34.340378 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb885b7-66a0-46ac-8923-6a20b613b764 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:34.342991 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb885b7-66a0-46ac-8923-6a20b613b764 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:34.347274 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb885b7-66a0-46ac-8923-6a20b613b764 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:34.353767 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a863d6-cd13-422a-8696-db89cf36ab7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=315f9034b39d43c98ef9064b7da5ff3e, user_id=64b055cfc9be4b4bb892f09eaba782ca, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:34.357824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a863d6-cd13-422a-8696-db89cf36ab7f admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:34.482185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 818/1632] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:10:34 2022] PUT /identity/v3/projects/315f9034b39d43c98ef9064b7da5ff3e/users/64b055cfc9be4b4bb892f09eaba782ca/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:34.483422 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a863d6-cd13-422a-8696-db89cf36ab7f admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:34.485245 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 815/1633] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:10:34 2022] POST /identity/v3/projects => generated 403 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:34.486369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb885b7-66a0-46ac-8923-6a20b613b764 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:34.557957 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:34.568492 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a863d6-cd13-422a-8696-db89cf36ab7f admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:34.578427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb885b7-66a0-46ac-8923-6a20b613b764 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:34.584752 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856ffd1-41ed-464f-b9fe-388b9c8f9efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:34.585544 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856ffd1-41ed-464f-b9fe-388b9c8f9efb admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:34.586053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856ffd1-41ed-464f-b9fe-388b9c8f9efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:34.586330 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856ffd1-41ed-464f-b9fe-388b9c8f9efb admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:34.587343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856ffd1-41ed-464f-b9fe-388b9c8f9efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:34.589478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856ffd1-41ed-464f-b9fe-388b9c8f9efb admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:34.614006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 816/1634] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:34.615905 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856ffd1-41ed-464f-b9fe-388b9c8f9efb admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:34.632095 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856ffd1-41ed-464f-b9fe-388b9c8f9efb admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:34.646729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b876e-7050-40e3-9a7e-a66fcf4418ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:34.655645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b876e-7050-40e3-9a7e-a66fcf4418ae admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:34.656494 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b876e-7050-40e3-9a7e-a66fcf4418ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:34.656995 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b876e-7050-40e3-9a7e-a66fcf4418ae admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:34.657545 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78b876e-7050-40e3-9a7e-a66fcf4418ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:34.660804 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78b876e-7050-40e3-9a7e-a66fcf4418ae admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:34.701192 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6abf78-0c55-4ec1-ad7e-b182023ebf57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:34.702971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6abf78-0c55-4ec1-ad7e-b182023ebf57 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:34.703273 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6abf78-0c55-4ec1-ad7e-b182023ebf57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:34.703532 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6abf78-0c55-4ec1-ad7e-b182023ebf57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:34.808894 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 817/1635] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:10:34 2022] POST /identity/v3/users => generated 308 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:34.811220 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b876e-7050-40e3-9a7e-a66fcf4418ae admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:34.845440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b876e-7050-40e3-9a7e-a66fcf4418ae admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:34.858526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa3975-5407-40b1-89ce-15f23ea03a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:34.860410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa3975-5407-40b1-89ce-15f23ea03a74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:34.865775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa3975-5407-40b1-89ce-15f23ea03a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:34.865964 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa3975-5407-40b1-89ce-15f23ea03a74 admin admin] PATH_INFO: `/v3/projects/315f9034b39d43c98ef9064b7da5ff3e/users/64b055cfc9be4b4bb892f09eaba782ca/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:34.870091 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa3975-5407-40b1-89ce-15f23ea03a74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=315f9034b39d43c98ef9064b7da5ff3e, user_id=64b055cfc9be4b4bb892f09eaba782ca, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:34.872868 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa3975-5407-40b1-89ce-15f23ea03a74 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:34.873618 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6abf78-0c55-4ec1-ad7e-b182023ebf57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:34.875723 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6abf78-0c55-4ec1-ad7e-b182023ebf57 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:34.892152 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 818/1636] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:10:34 2022] PUT /identity/v3/projects/315f9034b39d43c98ef9064b7da5ff3e/users/64b055cfc9be4b4bb892f09eaba782ca/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:34.893980 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa3975-5407-40b1-89ce-15f23ea03a74 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:34.927602 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 819/1637] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:10:34.933560 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c2b31-c197-42c1-988a-3c004d45cd31 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:34.933560 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c2b31-c197-42c1-988a-3c004d45cd31 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:34.933560 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c2b31-c197-42c1-988a-3c004d45cd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:34.963215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:34.975230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-130c2b31-c197-42c1-988a-3c004d45cd31 None None] MFA Rules not processed for user `64b055cfc9be4b4bb892f09eaba782ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:34.976645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-130c2b31-c197-42c1-988a-3c004d45cd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:35.041751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa3975-5407-40b1-89ce-15f23ea03a74 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:35.049218 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476f0da-1d8e-4cd5-b031-38d80e2e0dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:35.049915 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476f0da-1d8e-4cd5-b031-38d80e2e0dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:35.050198 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476f0da-1d8e-4cd5-b031-38d80e2e0dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:35.050470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476f0da-1d8e-4cd5-b031-38d80e2e0dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:35.051345 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476f0da-1d8e-4cd5-b031-38d80e2e0dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:35.053935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f476f0da-1d8e-4cd5-b031-38d80e2e0dc4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:35.073625 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 819/1638] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:35.078158 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476f0da-1d8e-4cd5-b031-38d80e2e0dc4 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:35.103974 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476f0da-1d8e-4cd5-b031-38d80e2e0dc4 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:35.113261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63724ca-ce76-4911-a97a-58f9152f878a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:35.114765 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63724ca-ce76-4911-a97a-58f9152f878a admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:35.115091 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63724ca-ce76-4911-a97a-58f9152f878a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:35.116712 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63724ca-ce76-4911-a97a-58f9152f878a admin admin] PATH_INFO: `/v3/projects/dc1da1cecc8b4b92bb130937faf001ec/users/ed3198d6fbf843fd851d170ade92c113/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:35.122825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 820/1639] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:35.151736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63724ca-ce76-4911-a97a-58f9152f878a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc1da1cecc8b4b92bb130937faf001ec, user_id=ed3198d6fbf843fd851d170ade92c113, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:35.155481 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63724ca-ce76-4911-a97a-58f9152f878a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:35.370871 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c2b31-c197-42c1-988a-3c004d45cd31 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:35.371402 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 820/1640] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:10:35 2022] PUT /identity/v3/projects/dc1da1cecc8b4b92bb130937faf001ec/users/ed3198d6fbf843fd851d170ade92c113/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:35.371402 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63724ca-ce76-4911-a97a-58f9152f878a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:35.372103 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c2b31-c197-42c1-988a-3c004d45cd31 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:35.417745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d5afe-7136-4b4d-9db1-8232d77c86c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:35.419155 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d5afe-7136-4b4d-9db1-8232d77c86c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:35.420685 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d5afe-7136-4b4d-9db1-8232d77c86c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:35.421150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d5afe-7136-4b4d-9db1-8232d77c86c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:35.431004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63724ca-ce76-4911-a97a-58f9152f878a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:35.438321 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083739e-6bac-45a8-95aa-1cc49d1d1b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:35.439460 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083739e-6bac-45a8-95aa-1cc49d1d1b9b admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:35.440131 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083739e-6bac-45a8-95aa-1cc49d1d1b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:35.440486 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083739e-6bac-45a8-95aa-1cc49d1d1b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:35.441433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1083739e-6bac-45a8-95aa-1cc49d1d1b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:35.444343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1083739e-6bac-45a8-95aa-1cc49d1d1b9b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:35.458243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 821/1641] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:10:35.462677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083739e-6bac-45a8-95aa-1cc49d1d1b9b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:35.477534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083739e-6bac-45a8-95aa-1cc49d1d1b9b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:35.488268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d48bd-0179-49e5-9739-810f5de7e972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:35.489205 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d48bd-0179-49e5-9739-810f5de7e972 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:35.489486 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d48bd-0179-49e5-9739-810f5de7e972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:35.489904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d48bd-0179-49e5-9739-810f5de7e972 admin admin] PATH_INFO: `/v3/projects/dc1da1cecc8b4b92bb130937faf001ec/users/ed3198d6fbf843fd851d170ade92c113/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:35.492527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d48bd-0179-49e5-9739-810f5de7e972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc1da1cecc8b4b92bb130937faf001ec, user_id=ed3198d6fbf843fd851d170ade92c113, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:35.495245 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d48bd-0179-49e5-9739-810f5de7e972 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:35.518116 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 822/1642] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:10:35 2022] PUT /identity/v3/projects/dc1da1cecc8b4b92bb130937faf001ec/users/ed3198d6fbf843fd851d170ade92c113/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:35.521337 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d5afe-7136-4b4d-9db1-8232d77c86c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:35.523037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d5afe-7136-4b4d-9db1-8232d77c86c7 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:35.539721 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 821/1643] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:35 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:10:36.093548 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d48bd-0179-49e5-9739-810f5de7e972 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:36.109322 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:36.216066 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d48bd-0179-49e5-9739-810f5de7e972 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:36.228115 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f190a-2359-4e4c-b34f-bb5c6daeec3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:36.229101 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f190a-2359-4e4c-b34f-bb5c6daeec3a service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:36.229599 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f190a-2359-4e4c-b34f-bb5c6daeec3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:36.230049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f190a-2359-4e4c-b34f-bb5c6daeec3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:36.404174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f190a-2359-4e4c-b34f-bb5c6daeec3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:36.412089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f190a-2359-4e4c-b34f-bb5c6daeec3a service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:36.429162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 823/1644] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:10:39.159943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d5afe-7136-4b4d-9db1-8232d77c86c7 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:39.166016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:39.185720 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d5afe-7136-4b4d-9db1-8232d77c86c7 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:39.400561 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf9aebf-4ac8-4206-8ad0-06ea13b2a402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:39.401464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9aebf-4ac8-4206-8ad0-06ea13b2a402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:39.401743 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9aebf-4ac8-4206-8ad0-06ea13b2a402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:39.402010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf9aebf-4ac8-4206-8ad0-06ea13b2a402 admin admin] PATH_INFO: `/v3/users/d8110cff925b49b8b8b3a44cc1891fcf` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:39.405435 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9aebf-4ac8-4206-8ad0-06ea13b2a402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8110cff925b49b8b8b3a44cc1891fcf)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:39.407548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: WARNING py.warnings [None req-0cf9aebf-4ac8-4206-8ad0-06ea13b2a402 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 15:10:39.407548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: warnings.warn(msg) Jul 16 15:10:39.407548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: Jul 16 15:10:39.409887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf9aebf-4ac8-4206-8ad0-06ea13b2a402 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:39.757349 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b134b-9285-47be-9d8e-b41f93f9bb52 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:39.757886 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b134b-9285-47be-9d8e-b41f93f9bb52 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:39.758805 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b134b-9285-47be-9d8e-b41f93f9bb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:39.759575 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-0cf9aebf-4ac8-4206-8ad0-06ea13b2a402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8110cff925b49b8b8b3a44cc1891fcf'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:39.849812 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-0cf9aebf-4ac8-4206-8ad0-06ea13b2a402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8110cff925b49b8b8b3a44cc1891fcf'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:39.889787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-0cf9aebf-4ac8-4206-8ad0-06ea13b2a402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8110cff925b49b8b8b3a44cc1891fcf'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:39.900788 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-bb1b134b-9285-47be-9d8e-b41f93f9bb52 None None] MFA Rules not processed for user `ed3198d6fbf843fd851d170ade92c113`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:39.909668 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bb1b134b-9285-47be-9d8e-b41f93f9bb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:39.992298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-0cf9aebf-4ac8-4206-8ad0-06ea13b2a402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8110cff925b49b8b8b3a44cc1891fcf'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:39.994161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-0cf9aebf-4ac8-4206-8ad0-06ea13b2a402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8110cff925b49b8b8b3a44cc1891fcf'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:40.015007 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 822/1645] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:10:39 2022] DELETE /identity/v3/users/d8110cff925b49b8b8b3a44cc1891fcf => generated 0 bytes in 853 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:40.022031 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a20b8-51be-4c61-93f9-433dac06dca9 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:40.023925 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a20b8-51be-4c61-93f9-433dac06dca9 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:40.026765 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a20b8-51be-4c61-93f9-433dac06dca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:40.121572 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-8b5a20b8-51be-4c61-93f9-433dac06dca9 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:40.129482 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-8b5a20b8-51be-4c61-93f9-433dac06dca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:40.257275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 824/1646] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:39 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 503 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:40.314688 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 823/1647] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:40.373377 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b134b-9285-47be-9d8e-b41f93f9bb52 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:40.375897 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bb1b134b-9285-47be-9d8e-b41f93f9bb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:40.502025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:40.507917 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b134b-9285-47be-9d8e-b41f93f9bb52 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:40.607160 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847059f-4c85-4439-85dc-920347e18a35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:40.609118 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847059f-4c85-4439-85dc-920347e18a35 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:40.609414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847059f-4c85-4439-85dc-920347e18a35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:40.610662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847059f-4c85-4439-85dc-920347e18a35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:40.638135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847059f-4c85-4439-85dc-920347e18a35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:40.640593 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847059f-4c85-4439-85dc-920347e18a35 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:40.663618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 825/1648] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:10:42.198119 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1827da-6ae4-4338-a81e-b5055a88541e None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:42.199216 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1827da-6ae4-4338-a81e-b5055a88541e None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:42.199636 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1827da-6ae4-4338-a81e-b5055a88541e None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:42.202113 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 824/1649] 10.222.0.27 () {54 vars in 921 bytes} [Sat Jul 16 15:10:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:10:42.215715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbf2b5-11fe-4f76-872a-48bb13826a52 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:42.216658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbf2b5-11fe-4f76-872a-48bb13826a52 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:42.217125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfbf2b5-11fe-4f76-872a-48bb13826a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:42.256072 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-bbfbf2b5-11fe-4f76-872a-48bb13826a52 None None] MFA Rules not processed for user `3a30cee766814c8685578819261705ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:42.257575 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bbfbf2b5-11fe-4f76-872a-48bb13826a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:42.482772 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 826/1650] 10.222.0.27 () {58 vars in 1018 bytes} [Sat Jul 16 15:10:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:42.530086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c8c23-bb0d-4ad1-998a-098be988b49b None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:42.530370 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c8c23-bb0d-4ad1-998a-098be988b49b None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:42.530804 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c8c23-bb0d-4ad1-998a-098be988b49b None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:42.532395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 825/1651] 10.222.0.27 () {54 vars in 960 bytes} [Sat Jul 16 15:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:10:42.545911 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62a53a-4482-437a-af08-f12c7ca8aba5 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:42.546435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62a53a-4482-437a-af08-f12c7ca8aba5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:42.546936 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62a53a-4482-437a-af08-f12c7ca8aba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:42.576552 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-4d62a53a-4482-437a-af08-f12c7ca8aba5 None None] MFA Rules not processed for user `3a30cee766814c8685578819261705ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:42.578701 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4d62a53a-4482-437a-af08-f12c7ca8aba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:42.594419 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 827/1652] 10.222.0.27 () {58 vars in 1057 bytes} [Sat Jul 16 15:10:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:42.597997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c8c23-bb0d-4ad1-998a-098be988b49b None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:42.634679 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c8c23-bb0d-4ad1-998a-098be988b49b None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:42.642561 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f92dd4-c95f-44f3-a8c8-d1706be986e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a30cee766814c8685578819261705ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:42.643578 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f92dd4-c95f-44f3-a8c8-d1706be986e8 service placement] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:42.644036 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f92dd4-c95f-44f3-a8c8-d1706be986e8 service placement] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:42.646336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f92dd4-c95f-44f3-a8c8-d1706be986e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:42.673243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f92dd4-c95f-44f3-a8c8-d1706be986e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:42.675656 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f92dd4-c95f-44f3-a8c8-d1706be986e8 service placement] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:42.693998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 826/1653] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:10:43.224580 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62a53a-4482-437a-af08-f12c7ca8aba5 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:43.226826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4d62a53a-4482-437a-af08-f12c7ca8aba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:43.269378 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62a53a-4482-437a-af08-f12c7ca8aba5 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:43.282095 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fc36e-5c17-44b5-8e19-a9e863190354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:43.283276 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fc36e-5c17-44b5-8e19-a9e863190354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:43.283602 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fc36e-5c17-44b5-8e19-a9e863190354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:43.285326 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fc36e-5c17-44b5-8e19-a9e863190354 admin admin] PATH_INFO: `/v3/projects/bbea395ccd654ee2af057e8ef3b5451c` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:43.286588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fc36e-5c17-44b5-8e19-a9e863190354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbea395ccd654ee2af057e8ef3b5451c)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:43.287714 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: WARNING py.warnings [None req-e11fc36e-5c17-44b5-8e19-a9e863190354 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 16 15:10:43.287714 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: warnings.warn(msg) Jul 16 15:10:43.287714 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: Jul 16 15:10:43.288939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11fc36e-5c17-44b5-8e19-a9e863190354 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:43.484664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e11fc36e-5c17-44b5-8e19-a9e863190354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbea395ccd654ee2af057e8ef3b5451c', 'request_id': 'req-e11fc36e-5c17-44b5-8e19-a9e863190354'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:43.504171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e11fc36e-5c17-44b5-8e19-a9e863190354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbea395ccd654ee2af057e8ef3b5451c', 'request_id': 'req-e11fc36e-5c17-44b5-8e19-a9e863190354'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:43.550256 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e11fc36e-5c17-44b5-8e19-a9e863190354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbea395ccd654ee2af057e8ef3b5451c', 'request_id': 'req-e11fc36e-5c17-44b5-8e19-a9e863190354'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:43.718762 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e11fc36e-5c17-44b5-8e19-a9e863190354 admin admin] The token cache is being invalidate because project bbea395ccd654ee2af057e8ef3b5451c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:10:43.719208 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e11fc36e-5c17-44b5-8e19-a9e863190354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:43.722680 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e11fc36e-5c17-44b5-8e19-a9e863190354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:43.725657 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 828/1654] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:10:43 2022] DELETE /identity/v3/projects/bbea395ccd654ee2af057e8ef3b5451c => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:43.730764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca117ed-0d27-4f3c-a245-4d9bbe4b792a None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:43.734057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca117ed-0d27-4f3c-a245-4d9bbe4b792a None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:43.734057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca117ed-0d27-4f3c-a245-4d9bbe4b792a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:43.759377 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-7ca117ed-0d27-4f3c-a245-4d9bbe4b792a None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:43.760818 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7ca117ed-0d27-4f3c-a245-4d9bbe4b792a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:43.778419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:43.912779 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 827/1655] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:43.921182 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fe775-7fc6-4622-a4e0-9a5da0eb44bc None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:43.921474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fe775-7fc6-4622-a4e0-9a5da0eb44bc None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:43.921756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fe775-7fc6-4622-a4e0-9a5da0eb44bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:43.979545 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-969fe775-7fc6-4622-a4e0-9a5da0eb44bc None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:43.980795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-969fe775-7fc6-4622-a4e0-9a5da0eb44bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:44.016492 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 829/1656] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:44.024601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca117ed-0d27-4f3c-a245-4d9bbe4b792a None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:44.041349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca117ed-0d27-4f3c-a245-4d9bbe4b792a None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:44.056353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33228861-aed1-422b-bdb5-4a60e947ec06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:44.058749 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33228861-aed1-422b-bdb5-4a60e947ec06 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:44.059280 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33228861-aed1-422b-bdb5-4a60e947ec06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:44.060089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33228861-aed1-422b-bdb5-4a60e947ec06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:44.061044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33228861-aed1-422b-bdb5-4a60e947ec06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:10:44.061361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33228861-aed1-422b-bdb5-4a60e947ec06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:44.070078 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33228861-aed1-422b-bdb5-4a60e947ec06 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:44.107826 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 828/1657] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:10:44.112439 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fe775-7fc6-4622-a4e0-9a5da0eb44bc None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:44.133098 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fe775-7fc6-4622-a4e0-9a5da0eb44bc None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:44.147712 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a3ce6-9f72-43b1-9862-0f1c50ad7019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:44.148761 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a3ce6-9f72-43b1-9862-0f1c50ad7019 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:44.149190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a3ce6-9f72-43b1-9862-0f1c50ad7019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:44.149621 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a3ce6-9f72-43b1-9862-0f1c50ad7019 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:44.150540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88a3ce6-9f72-43b1-9862-0f1c50ad7019 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:44.157802 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88a3ce6-9f72-43b1-9862-0f1c50ad7019 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:44.182718 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:44.295166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33228861-aed1-422b-bdb5-4a60e947ec06 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:44.303457 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 830/1658] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:10:44 2022] POST /identity/v3/projects => generated 373 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:44.310568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33228861-aed1-422b-bdb5-4a60e947ec06 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:44.512636 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598b2b4-8570-46b9-8534-3e7e2b561497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:44.512636 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598b2b4-8570-46b9-8534-3e7e2b561497 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:44.512923 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598b2b4-8570-46b9-8534-3e7e2b561497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:44.513003 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598b2b4-8570-46b9-8534-3e7e2b561497 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:44.519298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6598b2b4-8570-46b9-8534-3e7e2b561497 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:44.521970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6598b2b4-8570-46b9-8534-3e7e2b561497 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:44.670940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 829/1659] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:10:44 2022] POST /identity/v3/users => generated 287 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:44.675346 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a3ce6-9f72-43b1-9862-0f1c50ad7019 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:44.690407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a3ce6-9f72-43b1-9862-0f1c50ad7019 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:44.706436 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9842011-8ed8-4baa-99ac-d7a40a06655b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:44.708214 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9842011-8ed8-4baa-99ac-d7a40a06655b admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:44.714780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9842011-8ed8-4baa-99ac-d7a40a06655b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:44.714780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9842011-8ed8-4baa-99ac-d7a40a06655b admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:44.714780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9842011-8ed8-4baa-99ac-d7a40a06655b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:44.714780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9842011-8ed8-4baa-99ac-d7a40a06655b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:44.750854 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 831/1660] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:44.751451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598b2b4-8570-46b9-8534-3e7e2b561497 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:44.769377 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598b2b4-8570-46b9-8534-3e7e2b561497 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:44.787473 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2da0d-2124-45c5-a6c2-87db529f7d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:44.788329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2da0d-2124-45c5-a6c2-87db529f7d64 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:44.788800 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2da0d-2124-45c5-a6c2-87db529f7d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:44.789175 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2da0d-2124-45c5-a6c2-87db529f7d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:44.790051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2da0d-2124-45c5-a6c2-87db529f7d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:44.792132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2da0d-2124-45c5-a6c2-87db529f7d64 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:44.823275 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 830/1661] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:10:44.828618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9842011-8ed8-4baa-99ac-d7a40a06655b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:44.851376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9842011-8ed8-4baa-99ac-d7a40a06655b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:44.860449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd11f5-09b3-4e83-8406-57ab323848c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:44.861379 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd11f5-09b3-4e83-8406-57ab323848c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:44.861664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd11f5-09b3-4e83-8406-57ab323848c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:44.862137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd11f5-09b3-4e83-8406-57ab323848c3 admin admin] PATH_INFO: `/v3/projects/2e9fbc04b77342d4b8aaab12623fef45/users/d664be4c9f97435fb3b2b7fcfd9cbb32/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:44.900818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd11f5-09b3-4e83-8406-57ab323848c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e9fbc04b77342d4b8aaab12623fef45, user_id=d664be4c9f97435fb3b2b7fcfd9cbb32, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:44.903705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd11f5-09b3-4e83-8406-57ab323848c3 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:45.034771 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 832/1662] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:10:44 2022] PUT /identity/v3/projects/2e9fbc04b77342d4b8aaab12623fef45/users/d664be4c9f97435fb3b2b7fcfd9cbb32/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:45.050532 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8ea70-ece6-424c-bac5-db4a32d72bd7 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:45.050940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8ea70-ece6-424c-bac5-db4a32d72bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:45.052641 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8ea70-ece6-424c-bac5-db4a32d72bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:45.147575 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-6dd8ea70-ece6-424c-bac5-db4a32d72bd7 None None] MFA Rules not processed for user `d664be4c9f97435fb3b2b7fcfd9cbb32`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:45.153691 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-6dd8ea70-ece6-424c-bac5-db4a32d72bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:45.434102 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd11f5-09b3-4e83-8406-57ab323848c3 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:45.439697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d0fd11f5-09b3-4e83-8406-57ab323848c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:45.480376 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 831/1663] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 432 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:45.485437 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8ea70-ece6-424c-bac5-db4a32d72bd7 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:45.520510 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:45.633165 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd11f5-09b3-4e83-8406-57ab323848c3 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:45.645808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a9291-8e2d-487d-bb45-ac8d96ad71d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:45.647167 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a9291-8e2d-487d-bb45-ac8d96ad71d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:45.647453 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a9291-8e2d-487d-bb45-ac8d96ad71d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:45.647731 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a9291-8e2d-487d-bb45-ac8d96ad71d1 admin admin] PATH_INFO: `/v3/users/d8f7fb3933b3465bab5dd3f3a11e0317` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:45.652813 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a9291-8e2d-487d-bb45-ac8d96ad71d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8f7fb3933b3465bab5dd3f3a11e0317)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:45.660072 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a9291-8e2d-487d-bb45-ac8d96ad71d1 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:45.779905 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8ea70-ece6-424c-bac5-db4a32d72bd7 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:45.788934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faff0e0-5566-4748-bbfa-503218a424bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:45.790404 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faff0e0-5566-4748-bbfa-503218a424bd admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:45.790979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faff0e0-5566-4748-bbfa-503218a424bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:45.791541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faff0e0-5566-4748-bbfa-503218a424bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:45.793204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faff0e0-5566-4748-bbfa-503218a424bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:45.798756 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faff0e0-5566-4748-bbfa-503218a424bd admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:45.959952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 832/1664] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:10:45 2022] POST /identity/v3/projects => generated 373 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:45.983179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faff0e0-5566-4748-bbfa-503218a424bd admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:46.028995 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-137a9291-8e2d-487d-bb45-ac8d96ad71d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8f7fb3933b3465bab5dd3f3a11e0317'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:46.164939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-137a9291-8e2d-487d-bb45-ac8d96ad71d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8f7fb3933b3465bab5dd3f3a11e0317'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:46.166724 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-137a9291-8e2d-487d-bb45-ac8d96ad71d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8f7fb3933b3465bab5dd3f3a11e0317'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:46.167827 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-137a9291-8e2d-487d-bb45-ac8d96ad71d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8f7fb3933b3465bab5dd3f3a11e0317'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:46.204737 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-137a9291-8e2d-487d-bb45-ac8d96ad71d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8f7fb3933b3465bab5dd3f3a11e0317'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:46.278137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faff0e0-5566-4748-bbfa-503218a424bd admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:46.309390 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad19b0-430f-46d5-9d5e-a4c2b447b789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:46.312632 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 833/1665] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:10:45 2022] DELETE /identity/v3/users/d8f7fb3933b3465bab5dd3f3a11e0317 => generated 0 bytes in 881 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:46.319349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad19b0-430f-46d5-9d5e-a4c2b447b789 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:46.321006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad19b0-430f-46d5-9d5e-a4c2b447b789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:46.321519 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad19b0-430f-46d5-9d5e-a4c2b447b789 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:46.324511 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad19b0-430f-46d5-9d5e-a4c2b447b789 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:46.326742 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a47291-fb7a-4aec-9377-f3e4371bbcec None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:46.327291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a47291-fb7a-4aec-9377-f3e4371bbcec None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:46.331090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a47291-fb7a-4aec-9377-f3e4371bbcec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:46.338934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad19b0-430f-46d5-9d5e-a4c2b447b789 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:46.455832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-47a47291-fb7a-4aec-9377-f3e4371bbcec None None] MFA Rules not processed for user `19104226c2a64fcab522a5f2ade6fc3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:46.458976 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-47a47291-fb7a-4aec-9377-f3e4371bbcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:46.479317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 833/1666] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:10:45 2022] POST /identity/v3/users => generated 293 bytes in 512 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:46.482560 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad19b0-430f-46d5-9d5e-a4c2b447b789 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:46.486006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-d1ad19b0-430f-46d5-9d5e-a4c2b447b789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:46.577136 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad19b0-430f-46d5-9d5e-a4c2b447b789 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:46.592100 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:46.739636 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 834/1667] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:46.746559 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ad8da-34c8-48e6-9838-95b4c0cd03b5 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:46.746690 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ad8da-34c8-48e6-9838-95b4c0cd03b5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:46.746964 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ad8da-34c8-48e6-9838-95b4c0cd03b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:46.785430 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-829ad8da-34c8-48e6-9838-95b4c0cd03b5 None None] MFA Rules not processed for user `19104226c2a64fcab522a5f2ade6fc3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:46.792481 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-829ad8da-34c8-48e6-9838-95b4c0cd03b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:46.808305 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 835/1668] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:10:46.824015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ad8da-34c8-48e6-9838-95b4c0cd03b5 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:46.825772 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-829ad8da-34c8-48e6-9838-95b4c0cd03b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:46.941538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b14dc-4ab4-49d0-8f91-843dfef6df2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:46.952652 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b14dc-4ab4-49d0-8f91-843dfef6df2d admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:46.953255 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b14dc-4ab4-49d0-8f91-843dfef6df2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:46.953732 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b14dc-4ab4-49d0-8f91-843dfef6df2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:46.956180 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b14dc-4ab4-49d0-8f91-843dfef6df2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:46.965366 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b14dc-4ab4-49d0-8f91-843dfef6df2d admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:46.982726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ad8da-34c8-48e6-9838-95b4c0cd03b5 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:46.993939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd17e6-9eb9-43dc-b7dd-805d9ccf4b2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:46.993939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd17e6-9eb9-43dc-b7dd-805d9ccf4b2f service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:46.993939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd17e6-9eb9-43dc-b7dd-805d9ccf4b2f service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:46.993939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bd17e6-9eb9-43dc-b7dd-805d9ccf4b2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:47.003287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd17e6-9eb9-43dc-b7dd-805d9ccf4b2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:47.003782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 834/1669] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 524 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:47.006398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bd17e6-9eb9-43dc-b7dd-805d9ccf4b2f service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:47.009115 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b14dc-4ab4-49d0-8f91-843dfef6df2d admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:47.018369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-127b14dc-4ab4-49d0-8f91-843dfef6df2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:47.022232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 836/1670] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:10:47.023383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd17e6-9eb9-43dc-b7dd-805d9ccf4b2f service nova] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:47.036922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bd17e6-9eb9-43dc-b7dd-805d9ccf4b2f service nova] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:47.043369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127be142-8909-499b-a741-ff04d9a160f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:47.044755 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127be142-8909-499b-a741-ff04d9a160f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:47.045249 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127be142-8909-499b-a741-ff04d9a160f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:47.045887 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127be142-8909-499b-a741-ff04d9a160f5 admin admin] PATH_INFO: `/v3/projects/98e6d11d096a458d9fbc740229c14417/users/14c1668c56ae455ab1904df9e99445b2/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:47.055323 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b14dc-4ab4-49d0-8f91-843dfef6df2d admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:47.071654 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44a114-39f9-4df2-8480-4a8c690b13ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:47.072603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44a114-39f9-4df2-8480-4a8c690b13ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:47.072844 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44a114-39f9-4df2-8480-4a8c690b13ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:47.079933 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127be142-8909-499b-a741-ff04d9a160f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98e6d11d096a458d9fbc740229c14417, user_id=14c1668c56ae455ab1904df9e99445b2, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:47.080078 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44a114-39f9-4df2-8480-4a8c690b13ee admin admin] PATH_INFO: `/v3/users/e56835fbb6d14b5a8319d341440a03cd` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:47.080153 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127be142-8909-499b-a741-ff04d9a160f5 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:47.080217 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44a114-39f9-4df2-8480-4a8c690b13ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=e56835fbb6d14b5a8319d341440a03cd)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:47.080217 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44a114-39f9-4df2-8480-4a8c690b13ee admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:47.102572 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:47.188430 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 837/1671] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:10:47 2022] PUT /identity/v3/projects/98e6d11d096a458d9fbc740229c14417/users/14c1668c56ae455ab1904df9e99445b2/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:47.195460 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127be142-8909-499b-a741-ff04d9a160f5 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:47.394118 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127be142-8909-499b-a741-ff04d9a160f5 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:47.407176 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32688946-31cb-42db-9bd5-f77040ca75aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:47.407929 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32688946-31cb-42db-9bd5-f77040ca75aa admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:47.408208 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32688946-31cb-42db-9bd5-f77040ca75aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:47.408519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32688946-31cb-42db-9bd5-f77040ca75aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:47.410235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32688946-31cb-42db-9bd5-f77040ca75aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:47.411758 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32688946-31cb-42db-9bd5-f77040ca75aa admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:47.431936 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-9b44a114-39f9-4df2-8480-4a8c690b13ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e56835fbb6d14b5a8319d341440a03cd'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:47.448592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 838/1672] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:47.449607 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32688946-31cb-42db-9bd5-f77040ca75aa admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:47.472113 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32688946-31cb-42db-9bd5-f77040ca75aa admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:47.478563 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57f513-0e90-4947-8181-262df6751287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:47.480330 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57f513-0e90-4947-8181-262df6751287 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:47.482039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57f513-0e90-4947-8181-262df6751287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:47.482553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57f513-0e90-4947-8181-262df6751287 admin admin] PATH_INFO: `/v3/projects/98e6d11d096a458d9fbc740229c14417/users/14c1668c56ae455ab1904df9e99445b2/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:47.485169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57f513-0e90-4947-8181-262df6751287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98e6d11d096a458d9fbc740229c14417, user_id=14c1668c56ae455ab1904df9e99445b2, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:47.488714 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb57f513-0e90-4947-8181-262df6751287 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:47.509930 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 839/1673] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:10:47 2022] PUT /identity/v3/projects/98e6d11d096a458d9fbc740229c14417/users/14c1668c56ae455ab1904df9e99445b2/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:47.525679 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed50d3-a822-4706-91ac-715dce026e44 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:47.526496 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed50d3-a822-4706-91ac-715dce026e44 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:47.527053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed50d3-a822-4706-91ac-715dce026e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:47.537891 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-9b44a114-39f9-4df2-8480-4a8c690b13ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e56835fbb6d14b5a8319d341440a03cd'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:47.554375 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:47.564813 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-9b44a114-39f9-4df2-8480-4a8c690b13ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e56835fbb6d14b5a8319d341440a03cd'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:47.597495 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-16ed50d3-a822-4706-91ac-715dce026e44 None None] MFA Rules not processed for user `14c1668c56ae455ab1904df9e99445b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:47.600585 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-16ed50d3-a822-4706-91ac-715dce026e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:47.621296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-9b44a114-39f9-4df2-8480-4a8c690b13ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e56835fbb6d14b5a8319d341440a03cd'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:47.624212 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-9b44a114-39f9-4df2-8480-4a8c690b13ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e56835fbb6d14b5a8319d341440a03cd'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:47.638851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 835/1674] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:10:47 2022] DELETE /identity/v3/users/e56835fbb6d14b5a8319d341440a03cd => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:47.643786 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295423c3-1b0c-4fc3-b2f7-dd2f60b17863 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:47.648924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295423c3-1b0c-4fc3-b2f7-dd2f60b17863 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:47.650154 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295423c3-1b0c-4fc3-b2f7-dd2f60b17863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:47.769036 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-295423c3-1b0c-4fc3-b2f7-dd2f60b17863 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:47.771702 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-295423c3-1b0c-4fc3-b2f7-dd2f60b17863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:47.782359 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 840/1675] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:47.790948 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed50d3-a822-4706-91ac-715dce026e44 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:47.794998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-16ed50d3-a822-4706-91ac-715dce026e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:47.972532 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed50d3-a822-4706-91ac-715dce026e44 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:48.023158 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 836/1676] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:10:48.086135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295423c3-1b0c-4fc3-b2f7-dd2f60b17863 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:48.089985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-295423c3-1b0c-4fc3-b2f7-dd2f60b17863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:48.099920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03965dc-313f-4039-bd3f-59216bad3f67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:48.100621 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03965dc-313f-4039-bd3f-59216bad3f67 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:48.100906 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03965dc-313f-4039-bd3f-59216bad3f67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:48.101184 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03965dc-313f-4039-bd3f-59216bad3f67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:48.326567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295423c3-1b0c-4fc3-b2f7-dd2f60b17863 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:48.328556 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:48.336864 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03965dc-313f-4039-bd3f-59216bad3f67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:48.339948 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03965dc-313f-4039-bd3f-59216bad3f67 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:48.341233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c8d65-4ddb-4d57-a864-f909fe972778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:48.344529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c8d65-4ddb-4d57-a864-f909fe972778 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:48.345455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c8d65-4ddb-4d57-a864-f909fe972778 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:48.346141 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c8d65-4ddb-4d57-a864-f909fe972778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:48.356263 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 841/1677] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:47 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:10:48.358135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03965dc-313f-4039-bd3f-59216bad3f67 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:48.363433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d03965dc-313f-4039-bd3f-59216bad3f67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:48.391937 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c8d65-4ddb-4d57-a864-f909fe972778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:48.397218 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03965dc-313f-4039-bd3f-59216bad3f67 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:48.397998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c8d65-4ddb-4d57-a864-f909fe972778 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:48.417790 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f4456-1855-4f0b-b699-7bb92a6b49c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:48.422043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f4456-1855-4f0b-b699-7bb92a6b49c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:48.424547 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f4456-1855-4f0b-b699-7bb92a6b49c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:48.425161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f4456-1855-4f0b-b699-7bb92a6b49c4 admin admin] PATH_INFO: `/v3/projects/c7cd6660a7e34644944b849247efb37c` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:48.434210 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f4456-1855-4f0b-b699-7bb92a6b49c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7cd6660a7e34644944b849247efb37c)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:48.439917 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f4456-1855-4f0b-b699-7bb92a6b49c4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:48.450195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 837/1678] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:10:48.459244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb549ea7-b284-482d-b743-872d78689e59 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:48.460829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb549ea7-b284-482d-b743-872d78689e59 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:48.460829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb549ea7-b284-482d-b743-872d78689e59 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:48.464200 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 838/1679] 10.222.0.27 () {54 vars in 912 bytes} [Sat Jul 16 15:10:48 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:10:48.486780 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05ce1f-ce8b-4ddb-a190-88f8f9e81968 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:48.487524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05ce1f-ce8b-4ddb-a190-88f8f9e81968 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:48.488278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05ce1f-ce8b-4ddb-a190-88f8f9e81968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:48.647292 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-ec05ce1f-ce8b-4ddb-a190-88f8f9e81968 None None] MFA Rules not processed for user `6e372cc62af24664b01a967435eb8902`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:48.648662 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ec05ce1f-ce8b-4ddb-a190-88f8f9e81968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:48.669170 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d06f4456-1855-4f0b-b699-7bb92a6b49c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7cd6660a7e34644944b849247efb37c', 'request_id': 'req-d06f4456-1855-4f0b-b699-7bb92a6b49c4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:48.680153 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d06f4456-1855-4f0b-b699-7bb92a6b49c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7cd6660a7e34644944b849247efb37c', 'request_id': 'req-d06f4456-1855-4f0b-b699-7bb92a6b49c4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:48.722836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d06f4456-1855-4f0b-b699-7bb92a6b49c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7cd6660a7e34644944b849247efb37c', 'request_id': 'req-d06f4456-1855-4f0b-b699-7bb92a6b49c4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:48.725058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:48.845249 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d06f4456-1855-4f0b-b699-7bb92a6b49c4 admin admin] The token cache is being invalidate because project c7cd6660a7e34644944b849247efb37c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:10:48.846541 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d06f4456-1855-4f0b-b699-7bb92a6b49c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:48.851916 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d06f4456-1855-4f0b-b699-7bb92a6b49c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:48.852724 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 839/1680] 10.222.0.27 () {58 vars in 1009 bytes} [Sat Jul 16 15:10:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:48.874433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 842/1681] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:10:48 2022] DELETE /identity/v3/projects/c7cd6660a7e34644944b849247efb37c => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:48.877611 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05ce1f-ce8b-4ddb-a190-88f8f9e81968 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:48.964716 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f4456-1855-4f0b-b699-7bb92a6b49c4 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:48.967684 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d06f4456-1855-4f0b-b699-7bb92a6b49c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:49.026471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05ce1f-ce8b-4ddb-a190-88f8f9e81968 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:49.035230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4283026-9402-4a47-a8a7-e8b96e00d239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:49.047034 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f4456-1855-4f0b-b699-7bb92a6b49c4 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:49.052312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4283026-9402-4a47-a8a7-e8b96e00d239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:49.053015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4283026-9402-4a47-a8a7-e8b96e00d239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:49.053490 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4283026-9402-4a47-a8a7-e8b96e00d239 admin admin] PATH_INFO: `/v3/users/ed3198d6fbf843fd851d170ade92c113` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:49.055682 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4283026-9402-4a47-a8a7-e8b96e00d239 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed3198d6fbf843fd851d170ade92c113)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:49.061833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4283026-9402-4a47-a8a7-e8b96e00d239 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:49.083391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4915ba-cbc5-4290-8e60-aa3240715284 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:49.084639 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4915ba-cbc5-4290-8e60-aa3240715284 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:49.085162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4915ba-cbc5-4290-8e60-aa3240715284 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:49.085664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4915ba-cbc5-4290-8e60-aa3240715284 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:49.104204 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6c4915ba-cbc5-4290-8e60-aa3240715284 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:49.160343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4915ba-cbc5-4290-8e60-aa3240715284 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:49.167467 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4915ba-cbc5-4290-8e60-aa3240715284 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:49.205116 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 843/1682] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:10:49.378113 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a4283026-9402-4a47-a8a7-e8b96e00d239 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed3198d6fbf843fd851d170ade92c113'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:49.477754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a4283026-9402-4a47-a8a7-e8b96e00d239 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed3198d6fbf843fd851d170ade92c113'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:49.490145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a4283026-9402-4a47-a8a7-e8b96e00d239 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed3198d6fbf843fd851d170ade92c113'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:49.530710 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a4283026-9402-4a47-a8a7-e8b96e00d239 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed3198d6fbf843fd851d170ade92c113'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:49.531657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a4283026-9402-4a47-a8a7-e8b96e00d239 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed3198d6fbf843fd851d170ade92c113'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:49.545051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 840/1683] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:10:48 2022] DELETE /identity/v3/users/ed3198d6fbf843fd851d170ade92c113 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:50.223554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4915ba-cbc5-4290-8e60-aa3240715284 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:50.226193 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6c4915ba-cbc5-4290-8e60-aa3240715284 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:50.289170 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4915ba-cbc5-4290-8e60-aa3240715284 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:50.455130 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde641f6-0ee0-4e06-9d70-131cc5061988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:50.456038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde641f6-0ee0-4e06-9d70-131cc5061988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:50.456320 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde641f6-0ee0-4e06-9d70-131cc5061988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:50.456597 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde641f6-0ee0-4e06-9d70-131cc5061988 admin admin] PATH_INFO: `/v3/users/d664be4c9f97435fb3b2b7fcfd9cbb32` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:50.459361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde641f6-0ee0-4e06-9d70-131cc5061988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d664be4c9f97435fb3b2b7fcfd9cbb32)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:50.469777 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde641f6-0ee0-4e06-9d70-131cc5061988 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:50.788666 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fde641f6-0ee0-4e06-9d70-131cc5061988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd664be4c9f97435fb3b2b7fcfd9cbb32'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:50.866776 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fde641f6-0ee0-4e06-9d70-131cc5061988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd664be4c9f97435fb3b2b7fcfd9cbb32'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:50.867427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fde641f6-0ee0-4e06-9d70-131cc5061988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd664be4c9f97435fb3b2b7fcfd9cbb32'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:50.868045 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fde641f6-0ee0-4e06-9d70-131cc5061988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd664be4c9f97435fb3b2b7fcfd9cbb32'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:50.882599 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fde641f6-0ee0-4e06-9d70-131cc5061988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd664be4c9f97435fb3b2b7fcfd9cbb32'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:50.893799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:50.980651 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 844/1684] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:10:50 2022] DELETE /identity/v3/users/d664be4c9f97435fb3b2b7fcfd9cbb32 => generated 0 bytes in 759 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:50.985381 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c6643-f63f-4f22-ae7e-3534f2c5f2ba None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:50.986349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c6643-f63f-4f22-ae7e-3534f2c5f2ba None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:50.986818 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c6643-f63f-4f22-ae7e-3534f2c5f2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:51.049725 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-482c6643-f63f-4f22-ae7e-3534f2c5f2ba None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:51.052365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-482c6643-f63f-4f22-ae7e-3534f2c5f2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:51.181055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:51.252042 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 841/1685] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:51.267929 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde641f6-0ee0-4e06-9d70-131cc5061988 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:51.270488 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-fde641f6-0ee0-4e06-9d70-131cc5061988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:51.455252 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde641f6-0ee0-4e06-9d70-131cc5061988 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:51.474426 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4e41e-3a07-4d17-b046-6b3986aae536 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:51.476432 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4e41e-3a07-4d17-b046-6b3986aae536 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:51.480671 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4e41e-3a07-4d17-b046-6b3986aae536 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:51.481286 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4e41e-3a07-4d17-b046-6b3986aae536 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:51.511312 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4e41e-3a07-4d17-b046-6b3986aae536 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:51.513299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4e41e-3a07-4d17-b046-6b3986aae536 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:51.543992 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 845/1686] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:10:51.592811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c6643-f63f-4f22-ae7e-3534f2c5f2ba None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:51.597136 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-482c6643-f63f-4f22-ae7e-3534f2c5f2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:51.598851 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4e41e-3a07-4d17-b046-6b3986aae536 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:51.606743 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-67f4e41e-3a07-4d17-b046-6b3986aae536 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:51.640898 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4e41e-3a07-4d17-b046-6b3986aae536 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:51.659392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7daf6-9729-4091-a229-5af4868952f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:51.666311 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7daf6-9729-4091-a229-5af4868952f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:51.667404 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7daf6-9729-4091-a229-5af4868952f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:51.669114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7daf6-9729-4091-a229-5af4868952f4 admin admin] PATH_INFO: `/v3/projects/dc1da1cecc8b4b92bb130937faf001ec` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:51.678176 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7daf6-9729-4091-a229-5af4868952f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc1da1cecc8b4b92bb130937faf001ec)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:51.686700 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7daf6-9729-4091-a229-5af4868952f4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:51.836569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c6643-f63f-4f22-ae7e-3534f2c5f2ba None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:51.866161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d853a1c-06bb-4722-81a5-dc65d585e422 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a30cee766814c8685578819261705ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:51.868640 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d853a1c-06bb-4722-81a5-dc65d585e422 service placement] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:51.869483 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d853a1c-06bb-4722-81a5-dc65d585e422 service placement] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:51.870301 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d853a1c-06bb-4722-81a5-dc65d585e422 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:51.875401 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7d853a1c-06bb-4722-81a5-dc65d585e422 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:51.949671 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d853a1c-06bb-4722-81a5-dc65d585e422 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:51.963550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6cc7daf6-9729-4091-a229-5af4868952f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc1da1cecc8b4b92bb130937faf001ec', 'request_id': 'req-6cc7daf6-9729-4091-a229-5af4868952f4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:51.964146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d853a1c-06bb-4722-81a5-dc65d585e422 service placement] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:52.003489 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6cc7daf6-9729-4091-a229-5af4868952f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc1da1cecc8b4b92bb130937faf001ec', 'request_id': 'req-6cc7daf6-9729-4091-a229-5af4868952f4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:52.003575 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 842/1687] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:10:52.004111 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d853a1c-06bb-4722-81a5-dc65d585e422 service placement] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:52.008612 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7d853a1c-06bb-4722-81a5-dc65d585e422 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:52.012723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6cc7daf6-9729-4091-a229-5af4868952f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc1da1cecc8b4b92bb130937faf001ec', 'request_id': 'req-6cc7daf6-9729-4091-a229-5af4868952f4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:52.046290 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d853a1c-06bb-4722-81a5-dc65d585e422 service placement] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:52.112062 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6cc7daf6-9729-4091-a229-5af4868952f4 admin admin] The token cache is being invalidate because project dc1da1cecc8b4b92bb130937faf001ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:10:52.112360 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6cc7daf6-9729-4091-a229-5af4868952f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:52.114261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6cc7daf6-9729-4091-a229-5af4868952f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:52.127054 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 846/1688] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:10:51 2022] DELETE /identity/v3/projects/dc1da1cecc8b4b92bb130937faf001ec => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:52.127054 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7d66e-d574-4dae-b518-a5b59fe090b1 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:52.127054 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7d66e-d574-4dae-b518-a5b59fe090b1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:52.127054 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7d66e-d574-4dae-b518-a5b59fe090b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:52.234500 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5ff38-3f6c-49ab-b6ae-dd09272e37db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:52.241896 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5ff38-3f6c-49ab-b6ae-dd09272e37db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:52.241896 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5ff38-3f6c-49ab-b6ae-dd09272e37db admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:52.241896 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5ff38-3f6c-49ab-b6ae-dd09272e37db admin admin] PATH_INFO: `/v3/projects/f1efed993ab5428593567b31a0a6cbad` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:52.241896 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5ff38-3f6c-49ab-b6ae-dd09272e37db admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1efed993ab5428593567b31a0a6cbad)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:52.241896 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5ff38-3f6c-49ab-b6ae-dd09272e37db admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:52.269504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-70f7d66e-d574-4dae-b518-a5b59fe090b1 None None] MFA Rules not processed for user `2517f9ee6c3b4d52b44d5da510e7cbf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:52.276059 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-70f7d66e-d574-4dae-b518-a5b59fe090b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:52.406432 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:52.417178 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 847/1689] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:52.417329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-95d5ff38-3f6c-49ab-b6ae-dd09272e37db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1efed993ab5428593567b31a0a6cbad', 'request_id': 'req-95d5ff38-3f6c-49ab-b6ae-dd09272e37db'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:52.419883 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334cda0-f2b5-4a02-b460-c99f1887a280 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:52.420601 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334cda0-f2b5-4a02-b460-c99f1887a280 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:52.421298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334cda0-f2b5-4a02-b460-c99f1887a280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:52.446612 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-95d5ff38-3f6c-49ab-b6ae-dd09272e37db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1efed993ab5428593567b31a0a6cbad', 'request_id': 'req-95d5ff38-3f6c-49ab-b6ae-dd09272e37db'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:52.459518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-95d5ff38-3f6c-49ab-b6ae-dd09272e37db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1efed993ab5428593567b31a0a6cbad', 'request_id': 'req-95d5ff38-3f6c-49ab-b6ae-dd09272e37db'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:52.485635 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-1334cda0-f2b5-4a02-b460-c99f1887a280 None None] MFA Rules not processed for user `096fd1b2e2b4482da1026d3fb7e192c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:52.489821 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1334cda0-f2b5-4a02-b460-c99f1887a280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:52.564741 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-95d5ff38-3f6c-49ab-b6ae-dd09272e37db admin admin] The token cache is being invalidate because project f1efed993ab5428593567b31a0a6cbad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:10:52.565042 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-95d5ff38-3f6c-49ab-b6ae-dd09272e37db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:52.568994 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-95d5ff38-3f6c-49ab-b6ae-dd09272e37db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:52.581681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 843/1690] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:10:52 2022] DELETE /identity/v3/projects/f1efed993ab5428593567b31a0a6cbad => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:52.586992 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63466436-6509-46ae-b84a-e0da2279d7c4 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:52.592896 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63466436-6509-46ae-b84a-e0da2279d7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:52.592896 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63466436-6509-46ae-b84a-e0da2279d7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:52.635433 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:52.666317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-63466436-6509-46ae-b84a-e0da2279d7c4 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:52.668632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-63466436-6509-46ae-b84a-e0da2279d7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:52.676538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 848/1691] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:52.676538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa917e0-cf4c-4bc3-8c49-0c17caa81ae3 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:52.676538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa917e0-cf4c-4bc3-8c49-0c17caa81ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:52.676538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa917e0-cf4c-4bc3-8c49-0c17caa81ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:52.748910 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-8fa917e0-cf4c-4bc3-8c49-0c17caa81ae3 None None] MFA Rules not processed for user `2517f9ee6c3b4d52b44d5da510e7cbf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:52.750244 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-8fa917e0-cf4c-4bc3-8c49-0c17caa81ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:52.851312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 844/1692] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:52.852803 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5e1dd-62d6-4a4a-869d-757767163e74 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:52.854767 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5e1dd-62d6-4a4a-869d-757767163e74 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:52.855336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5e1dd-62d6-4a4a-869d-757767163e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:52.931548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-2cc5e1dd-62d6-4a4a-869d-757767163e74 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:52.933268 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-2cc5e1dd-62d6-4a4a-869d-757767163e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:52.981879 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 845/1693] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:52.981879 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5e1dd-62d6-4a4a-869d-757767163e74 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:52.991276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-2cc5e1dd-62d6-4a4a-869d-757767163e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:53.014922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 849/1694] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:53.018062 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0137c-6795-499b-b3f2-fe73e9f9c94e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:53.019148 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0137c-6795-499b-b3f2-fe73e9f9c94e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:53.019913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0137c-6795-499b-b3f2-fe73e9f9c94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:53.076295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-91d0137c-6795-499b-b3f2-fe73e9f9c94e None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:53.091611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-91d0137c-6795-499b-b3f2-fe73e9f9c94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:53.117124 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 850/1695] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:53.120980 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a821b4-863d-4893-b095-ba9686708c51 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:53.121834 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a821b4-863d-4893-b095-ba9686708c51 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:53.122416 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a821b4-863d-4893-b095-ba9686708c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:53.130381 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5e1dd-62d6-4a4a-869d-757767163e74 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:53.137720 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bce9c-2d7e-4149-9449-8b4fff172c71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:53.138643 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bce9c-2d7e-4149-9449-8b4fff172c71 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:53.138866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bce9c-2d7e-4149-9449-8b4fff172c71 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:53.139342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bce9c-2d7e-4149-9449-8b4fff172c71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:53.152218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5bce9c-2d7e-4149-9449-8b4fff172c71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:53.154717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5bce9c-2d7e-4149-9449-8b4fff172c71 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:53.176715 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 846/1696] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:10:53.180528 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9e449-1110-427b-9d34-bf832294be73 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:53.180822 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9e449-1110-427b-9d34-bf832294be73 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:53.181113 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9e449-1110-427b-9d34-bf832294be73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:53.191601 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a1a821b4-863d-4893-b095-ba9686708c51 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:53.193509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a1a821b4-863d-4893-b095-ba9686708c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:53.236593 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 851/1697] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:53.237860 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a821b4-863d-4893-b095-ba9686708c51 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:53.240793 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a1a821b4-863d-4893-b095-ba9686708c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:53.259713 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-2ca9e449-1110-427b-9d34-bf832294be73 None None] MFA Rules not processed for user `9ee0106af0894acb911831433ca22caf`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:53.267062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-2ca9e449-1110-427b-9d34-bf832294be73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:53.277195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a821b4-863d-4893-b095-ba9686708c51 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:53.290766 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74235d6c-c36c-4364-8e66-7cc6169bd316 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:53.291830 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74235d6c-c36c-4364-8e66-7cc6169bd316 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:53.292401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74235d6c-c36c-4364-8e66-7cc6169bd316 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:53.292901 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74235d6c-c36c-4364-8e66-7cc6169bd316 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:53.298811 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-74235d6c-c36c-4364-8e66-7cc6169bd316 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:53.330147 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74235d6c-c36c-4364-8e66-7cc6169bd316 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:53.333190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74235d6c-c36c-4364-8e66-7cc6169bd316 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:53.358534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 852/1698] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:10:53.359553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74235d6c-c36c-4364-8e66-7cc6169bd316 service nova] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:53.377039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74235d6c-c36c-4364-8e66-7cc6169bd316 service nova] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:53.383391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6294c-d328-43ab-b5a2-504af031c473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:53.384366 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6294c-d328-43ab-b5a2-504af031c473 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:53.384661 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6294c-d328-43ab-b5a2-504af031c473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:53.385770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6294c-d328-43ab-b5a2-504af031c473 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:53.386696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6294c-d328-43ab-b5a2-504af031c473 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:10:53.387113 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6294c-d328-43ab-b5a2-504af031c473 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:53.388922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6294c-d328-43ab-b5a2-504af031c473 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:53.411687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 853/1699] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:10:53.413549 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6294c-d328-43ab-b5a2-504af031c473 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:53.419202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-51f6294c-d328-43ab-b5a2-504af031c473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:53.454368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6294c-d328-43ab-b5a2-504af031c473 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:53.486126 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 847/1700] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:53.487060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9e449-1110-427b-9d34-bf832294be73 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:53.536013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebe984-2977-4364-99cd-3e78af21e1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:53.536013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebe984-2977-4364-99cd-3e78af21e1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:53.536013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebe984-2977-4364-99cd-3e78af21e1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:53.536013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebe984-2977-4364-99cd-3e78af21e1a2 admin admin] PATH_INFO: `/v3/projects/2e9fbc04b77342d4b8aaab12623fef45` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:53.546041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bebe984-2977-4364-99cd-3e78af21e1a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e9fbc04b77342d4b8aaab12623fef45)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:53.556030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bebe984-2977-4364-99cd-3e78af21e1a2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:53.560806 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9e449-1110-427b-9d34-bf832294be73 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:53.566781 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87c5fa-27dd-4723-921d-cfd4a5a0b1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:53.569116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87c5fa-27dd-4723-921d-cfd4a5a0b1e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:53.569116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87c5fa-27dd-4723-921d-cfd4a5a0b1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:53.569589 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87c5fa-27dd-4723-921d-cfd4a5a0b1e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:53.570787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87c5fa-27dd-4723-921d-cfd4a5a0b1e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:10:53.571283 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87c5fa-27dd-4723-921d-cfd4a5a0b1e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:53.573624 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87c5fa-27dd-4723-921d-cfd4a5a0b1e4 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:53.612127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 848/1701] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:10:53.615942 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87c5fa-27dd-4723-921d-cfd4a5a0b1e4 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:53.622504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:53.628707 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87c5fa-27dd-4723-921d-cfd4a5a0b1e4 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:53.635247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a2e65-0182-4ade-a333-db1cf6c6af8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:53.636176 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a2e65-0182-4ade-a333-db1cf6c6af8e admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:53.636635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a2e65-0182-4ade-a333-db1cf6c6af8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:53.637049 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a2e65-0182-4ade-a333-db1cf6c6af8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:53.637930 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a2e65-0182-4ade-a333-db1cf6c6af8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:53.640183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a2e65-0182-4ade-a333-db1cf6c6af8e admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:53.769298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:53.773038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5bebe984-2977-4364-99cd-3e78af21e1a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e9fbc04b77342d4b8aaab12623fef45', 'request_id': 'req-5bebe984-2977-4364-99cd-3e78af21e1a2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:53.773522 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 849/1702] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:10:53 2022] POST /identity/v3/projects => generated 381 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:53.774502 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a2e65-0182-4ade-a333-db1cf6c6af8e admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:53.790371 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5bebe984-2977-4364-99cd-3e78af21e1a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e9fbc04b77342d4b8aaab12623fef45', 'request_id': 'req-5bebe984-2977-4364-99cd-3e78af21e1a2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:53.807368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5bebe984-2977-4364-99cd-3e78af21e1a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e9fbc04b77342d4b8aaab12623fef45', 'request_id': 'req-5bebe984-2977-4364-99cd-3e78af21e1a2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:53.905719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5bebe984-2977-4364-99cd-3e78af21e1a2 admin admin] The token cache is being invalidate because project 2e9fbc04b77342d4b8aaab12623fef45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:10:53.906108 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5bebe984-2977-4364-99cd-3e78af21e1a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:53.908228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5bebe984-2977-4364-99cd-3e78af21e1a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:53.918423 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 854/1703] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:10:53 2022] DELETE /identity/v3/projects/2e9fbc04b77342d4b8aaab12623fef45 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:53.920506 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebe984-2977-4364-99cd-3e78af21e1a2 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:53.924907 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5bebe984-2977-4364-99cd-3e78af21e1a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:53.997422 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a2e65-0182-4ade-a333-db1cf6c6af8e admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:54.029176 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02a212-e261-4aab-894d-a5333a579d8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:54.030307 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02a212-e261-4aab-894d-a5333a579d8c service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:54.035812 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02a212-e261-4aab-894d-a5333a579d8c service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:54.036439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02a212-e261-4aab-894d-a5333a579d8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:54.049150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-9e02a212-e261-4aab-894d-a5333a579d8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:54.249000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebe984-2977-4364-99cd-3e78af21e1a2 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:54.347270 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02a212-e261-4aab-894d-a5333a579d8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:54.350208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02a212-e261-4aab-894d-a5333a579d8c service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:54.365757 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 850/1704] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:10:54.366725 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02a212-e261-4aab-894d-a5333a579d8c service nova] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:54.370142 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-9e02a212-e261-4aab-894d-a5333a579d8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:54.468090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d90f6-5e21-4904-a4f8-276885b791a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:54.469015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d90f6-5e21-4904-a4f8-276885b791a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:54.469418 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d90f6-5e21-4904-a4f8-276885b791a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:54.469825 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d90f6-5e21-4904-a4f8-276885b791a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:54.470951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d90f6-5e21-4904-a4f8-276885b791a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:54.481557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d90f6-5e21-4904-a4f8-276885b791a2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:54.593138 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02a212-e261-4aab-894d-a5333a579d8c service nova] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:54.616947 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7176b2-9920-43b4-a784-344799febe7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:54.616947 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7176b2-9920-43b4-a784-344799febe7c admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:54.616947 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7176b2-9920-43b4-a784-344799febe7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:54.616947 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7176b2-9920-43b4-a784-344799febe7c admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:54.617358 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7176b2-9920-43b4-a784-344799febe7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:54.622750 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 855/1705] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:10:53 2022] POST /identity/v3/projects => generated 367 bytes in 701 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:54.622750 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d90f6-5e21-4904-a4f8-276885b791a2 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:54.625695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7176b2-9920-43b4-a784-344799febe7c admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:54.628274 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3a6d90f6-5e21-4904-a4f8-276885b791a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:54.768176 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 851/1706] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:10:54 2022] POST /identity/v3/users => generated 291 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:54.776276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7176b2-9920-43b4-a784-344799febe7c admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:54.812644 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7176b2-9920-43b4-a784-344799febe7c admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:54.826166 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d90f6-5e21-4904-a4f8-276885b791a2 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:54.838129 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738da65-8382-4114-aff3-525bf4ea605c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:54.840347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738da65-8382-4114-aff3-525bf4ea605c admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:54.840702 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738da65-8382-4114-aff3-525bf4ea605c admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:54.842929 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738da65-8382-4114-aff3-525bf4ea605c admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:54.843554 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738da65-8382-4114-aff3-525bf4ea605c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:54.844863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6cec24-8148-4296-b6df-3501dfaab97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:54.846760 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cec24-8148-4296-b6df-3501dfaab97d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:54.848174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3738da65-8382-4114-aff3-525bf4ea605c admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:54.848609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cec24-8148-4296-b6df-3501dfaab97d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:54.849232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cec24-8148-4296-b6df-3501dfaab97d admin admin] PATH_INFO: `/v3/users/14c1668c56ae455ab1904df9e99445b2` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:54.863172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6cec24-8148-4296-b6df-3501dfaab97d admin admin] RBAC: Authorizing `identity:delete_user(user_id=14c1668c56ae455ab1904df9e99445b2)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:54.873978 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:54.876897 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6cec24-8148-4296-b6df-3501dfaab97d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:54.903698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:55.030969 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 852/1707] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:10:54 2022] POST /identity/v3/users => generated 284 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:55.032975 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647965f-2b66-40ac-8304-a3ca6bc98084 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:55.033390 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647965f-2b66-40ac-8304-a3ca6bc98084 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:55.034734 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647965f-2b66-40ac-8304-a3ca6bc98084 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:55.037294 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 853/1708] 10.222.0.27 () {54 vars in 912 bytes} [Sat Jul 16 15:10:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:10:55.038354 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647965f-2b66-40ac-8304-a3ca6bc98084 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:55.058346 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647965f-2b66-40ac-8304-a3ca6bc98084 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:55.065924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43626f50-01b9-4b7a-a9f5-1390c8dc158e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:55.068345 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43626f50-01b9-4b7a-a9f5-1390c8dc158e admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:55.069067 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43626f50-01b9-4b7a-a9f5-1390c8dc158e admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:55.069946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43626f50-01b9-4b7a-a9f5-1390c8dc158e admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:55.071300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43626f50-01b9-4b7a-a9f5-1390c8dc158e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:55.075910 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43626f50-01b9-4b7a-a9f5-1390c8dc158e admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:55.099210 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 854/1709] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:55.102647 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43626f50-01b9-4b7a-a9f5-1390c8dc158e admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:55.133855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43626f50-01b9-4b7a-a9f5-1390c8dc158e admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:55.142462 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c019f-d176-49a0-b057-01699de72138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:55.143478 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c019f-d176-49a0-b057-01699de72138 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:55.145564 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c019f-d176-49a0-b057-01699de72138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:55.146050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c019f-d176-49a0-b057-01699de72138 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:55.156477 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c019f-d176-49a0-b057-01699de72138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:55.156477 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c019f-d176-49a0-b057-01699de72138 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:55.173382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 855/1710] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:55.173382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c39529-9dc7-4c03-8bd3-1449a14ab732 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:55.173382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c39529-9dc7-4c03-8bd3-1449a14ab732 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:55.173809 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c39529-9dc7-4c03-8bd3-1449a14ab732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:55.193647 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1f6cec24-8148-4296-b6df-3501dfaab97d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14c1668c56ae455ab1904df9e99445b2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:55.198830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-64c39529-9dc7-4c03-8bd3-1449a14ab732 None None] MFA Rules not processed for user `6e372cc62af24664b01a967435eb8902`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:55.200934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-64c39529-9dc7-4c03-8bd3-1449a14ab732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:55.299355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 856/1711] 10.222.0.27 () {58 vars in 1009 bytes} [Sat Jul 16 15:10:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:55.302199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c39529-9dc7-4c03-8bd3-1449a14ab732 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:55.332398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c39529-9dc7-4c03-8bd3-1449a14ab732 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:55.343176 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0034a63-3ae3-4218-9874-cb98feb341f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:55.344443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0034a63-3ae3-4218-9874-cb98feb341f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:55.345052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0034a63-3ae3-4218-9874-cb98feb341f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:55.345553 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0034a63-3ae3-4218-9874-cb98feb341f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:55.347084 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0034a63-3ae3-4218-9874-cb98feb341f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:55.350427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0034a63-3ae3-4218-9874-cb98feb341f7 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:55.366157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1f6cec24-8148-4296-b6df-3501dfaab97d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14c1668c56ae455ab1904df9e99445b2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:55.368179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1f6cec24-8148-4296-b6df-3501dfaab97d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14c1668c56ae455ab1904df9e99445b2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:55.370194 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1f6cec24-8148-4296-b6df-3501dfaab97d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14c1668c56ae455ab1904df9e99445b2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:55.372487 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 857/1712] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:55.374269 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0034a63-3ae3-4218-9874-cb98feb341f7 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:55.380752 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-f0034a63-3ae3-4218-9874-cb98feb341f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:55.397711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1f6cec24-8148-4296-b6df-3501dfaab97d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14c1668c56ae455ab1904df9e99445b2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:55.429618 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0034a63-3ae3-4218-9874-cb98feb341f7 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:55.435331 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 856/1713] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:10:54 2022] DELETE /identity/v3/users/14c1668c56ae455ab1904df9e99445b2 => generated 0 bytes in 815 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:55.436728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6cec24-8148-4296-b6df-3501dfaab97d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:55.438474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1f6cec24-8148-4296-b6df-3501dfaab97d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:55.460087 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628644a-b626-4d4f-b7e7-beca1a4f4a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:55.462183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628644a-b626-4d4f-b7e7-beca1a4f4a20 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:55.462527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628644a-b626-4d4f-b7e7-beca1a4f4a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:55.463033 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628644a-b626-4d4f-b7e7-beca1a4f4a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:55.466865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628644a-b626-4d4f-b7e7-beca1a4f4a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:55.474673 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628644a-b626-4d4f-b7e7-beca1a4f4a20 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:55.547641 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6cec24-8148-4296-b6df-3501dfaab97d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:55.548375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 858/1714] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:55.551847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628644a-b626-4d4f-b7e7-beca1a4f4a20 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:55.563800 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520b0d1-9e69-4254-96a9-d37e2f3d492d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:55.564176 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5628644a-b626-4d4f-b7e7-beca1a4f4a20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:55.565246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520b0d1-9e69-4254-96a9-d37e2f3d492d service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:55.566177 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520b0d1-9e69-4254-96a9-d37e2f3d492d service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:55.567478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520b0d1-9e69-4254-96a9-d37e2f3d492d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:55.572992 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-8520b0d1-9e69-4254-96a9-d37e2f3d492d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:55.591278 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520b0d1-9e69-4254-96a9-d37e2f3d492d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:55.592942 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520b0d1-9e69-4254-96a9-d37e2f3d492d service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:55.604204 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 857/1715] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:10:55.605286 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520b0d1-9e69-4254-96a9-d37e2f3d492d service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:55.624865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628644a-b626-4d4f-b7e7-beca1a4f4a20 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:55.931640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520b0d1-9e69-4254-96a9-d37e2f3d492d service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:55.958011 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02c6b7-da50-4492-8c44-d71a6e3ef326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:55.958619 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02c6b7-da50-4492-8c44-d71a6e3ef326 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:55.959179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02c6b7-da50-4492-8c44-d71a6e3ef326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:55.959333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02c6b7-da50-4492-8c44-d71a6e3ef326 admin admin] PATH_INFO: `/v3/projects/5401441f79af467a90cda0501aa0d9ca/users/038884289cd74095bde16ab0f1d1834b/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:55.992218 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c02c6b7-da50-4492-8c44-d71a6e3ef326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5401441f79af467a90cda0501aa0d9ca, user_id=038884289cd74095bde16ab0f1d1834b, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:55.995455 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c02c6b7-da50-4492-8c44-d71a6e3ef326 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:55.999221 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04feb1b8-febb-4d3f-8524-ea482d53dea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:56.002379 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04feb1b8-febb-4d3f-8524-ea482d53dea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:56.002940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04feb1b8-febb-4d3f-8524-ea482d53dea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:56.003441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04feb1b8-febb-4d3f-8524-ea482d53dea2 admin admin] PATH_INFO: `/v3/projects/de343aaf089b426184e5972f73a46b3e/users/81098bcea7884326b7f17258df7ea49b/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:56.042165 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04feb1b8-febb-4d3f-8524-ea482d53dea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de343aaf089b426184e5972f73a46b3e, user_id=81098bcea7884326b7f17258df7ea49b, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:56.043956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04feb1b8-febb-4d3f-8524-ea482d53dea2 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:56.154591 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 858/1716] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:10:55 2022] PUT /identity/v3/projects/5401441f79af467a90cda0501aa0d9ca/users/038884289cd74095bde16ab0f1d1834b/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:56.177975 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4abd1-fc48-4c7b-a267-8107bc951fb3 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:56.178445 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4abd1-fc48-4c7b-a267-8107bc951fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:56.178950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4abd1-fc48-4c7b-a267-8107bc951fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:56.253648 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 859/1717] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:10:55 2022] PUT /identity/v3/projects/de343aaf089b426184e5972f73a46b3e/users/81098bcea7884326b7f17258df7ea49b/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:56.258083 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02465f28-366b-4a69-99ea-6e5037209791 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:56.258328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02465f28-366b-4a69-99ea-6e5037209791 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:56.261812 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02465f28-366b-4a69-99ea-6e5037209791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:56.328687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:56.335575 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ddc4abd1-fc48-4c7b-a267-8107bc951fb3 None None] MFA Rules not processed for user `038884289cd74095bde16ab0f1d1834b`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:56.341318 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ddc4abd1-fc48-4c7b-a267-8107bc951fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:56.357076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-02465f28-366b-4a69-99ea-6e5037209791 None None] MFA Rules not processed for user `81098bcea7884326b7f17258df7ea49b`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:56.358379 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-02465f28-366b-4a69-99ea-6e5037209791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:56.458637 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:56.662755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 860/1718] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:56.671917 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02465f28-366b-4a69-99ea-6e5037209791 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:56.757202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 859/1719] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:56 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 581 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:10:56.787843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4abd1-fc48-4c7b-a267-8107bc951fb3 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:56.790116 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ddc4abd1-fc48-4c7b-a267-8107bc951fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:56.979823 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4abd1-fc48-4c7b-a267-8107bc951fb3 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:56.990559 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16316738-abae-4ef8-90f4-d330743a28ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:56.992186 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16316738-abae-4ef8-90f4-d330743a28ba service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:56.993786 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16316738-abae-4ef8-90f4-d330743a28ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:56.995641 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16316738-abae-4ef8-90f4-d330743a28ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:57.023486 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16316738-abae-4ef8-90f4-d330743a28ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:57.027551 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16316738-abae-4ef8-90f4-d330743a28ba service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:57.064859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 860/1720] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:10:56 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:10:57.066136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16316738-abae-4ef8-90f4-d330743a28ba service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:57.072921 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-16316738-abae-4ef8-90f4-d330743a28ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:57.180586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02465f28-366b-4a69-99ea-6e5037209791 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:57.215532 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e3ec3-b29a-4140-975a-39d5de67667d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:57.217445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e3ec3-b29a-4140-975a-39d5de67667d admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:57.217445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e3ec3-b29a-4140-975a-39d5de67667d admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:57.217785 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e3ec3-b29a-4140-975a-39d5de67667d admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:57.218813 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e3ec3-b29a-4140-975a-39d5de67667d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:57.230489 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e3ec3-b29a-4140-975a-39d5de67667d admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:57.391806 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 861/1721] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:10:56 2022] POST /identity/v3/projects => generated 381 bytes in 723 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:57.396835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e3ec3-b29a-4140-975a-39d5de67667d admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:57.478022 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16316738-abae-4ef8-90f4-d330743a28ba service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:57.759769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:57.809680 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959fed5-1f75-429d-a686-633c14f49c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:57.813036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959fed5-1f75-429d-a686-633c14f49c1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:57.813373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959fed5-1f75-429d-a686-633c14f49c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:57.813666 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959fed5-1f75-429d-a686-633c14f49c1f admin admin] PATH_INFO: `/v3/projects/98e6d11d096a458d9fbc740229c14417` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:57.819067 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959fed5-1f75-429d-a686-633c14f49c1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=98e6d11d096a458d9fbc740229c14417)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:57.824914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959fed5-1f75-429d-a686-633c14f49c1f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:57.865560 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e3ec3-b29a-4140-975a-39d5de67667d admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:57.908199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451ec8a-f37c-4429-a0c1-15ecd0a47b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:57.909375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451ec8a-f37c-4429-a0c1-15ecd0a47b44 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:57.909879 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451ec8a-f37c-4429-a0c1-15ecd0a47b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:57.910919 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451ec8a-f37c-4429-a0c1-15ecd0a47b44 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:57.912298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451ec8a-f37c-4429-a0c1-15ecd0a47b44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:57.935593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451ec8a-f37c-4429-a0c1-15ecd0a47b44 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:58.057671 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 862/1722] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:10:57 2022] POST /identity/v3/users => generated 297 bytes in 661 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:58.058628 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8959fed5-1f75-429d-a686-633c14f49c1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98e6d11d096a458d9fbc740229c14417', 'request_id': 'req-8959fed5-1f75-429d-a686-633c14f49c1f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:58.059901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f8858-22e1-4c9a-ac08-41b1a1555116 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:58.063009 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f8858-22e1-4c9a-ac08-41b1a1555116 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:58.064173 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f8858-22e1-4c9a-ac08-41b1a1555116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:58.086112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8959fed5-1f75-429d-a686-633c14f49c1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98e6d11d096a458d9fbc740229c14417', 'request_id': 'req-8959fed5-1f75-429d-a686-633c14f49c1f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:58.105343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8959fed5-1f75-429d-a686-633c14f49c1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98e6d11d096a458d9fbc740229c14417', 'request_id': 'req-8959fed5-1f75-429d-a686-633c14f49c1f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:58.152414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-af3f8858-22e1-4c9a-ac08-41b1a1555116 None None] MFA Rules not processed for user `2517f9ee6c3b4d52b44d5da510e7cbf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:58.158811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-af3f8858-22e1-4c9a-ac08-41b1a1555116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:58.273088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8959fed5-1f75-429d-a686-633c14f49c1f admin admin] The token cache is being invalidate because project 98e6d11d096a458d9fbc740229c14417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:10:58.273088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8959fed5-1f75-429d-a686-633c14f49c1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:58.275471 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8959fed5-1f75-429d-a686-633c14f49c1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:10:58.278046 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 861/1723] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:10:57 2022] DELETE /identity/v3/projects/98e6d11d096a458d9fbc740229c14417 => generated 0 bytes in 1212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:58.285950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959fed5-1f75-429d-a686-633c14f49c1f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:58.312985 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-8959fed5-1f75-429d-a686-633c14f49c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:58.368945 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:58.471415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 863/1724] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:58.477171 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb3c66-957a-4775-9056-6be276161232 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:58.478032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb3c66-957a-4775-9056-6be276161232 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:58.481193 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb3c66-957a-4775-9056-6be276161232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:58.640383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959fed5-1f75-429d-a686-633c14f49c1f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:58.660483 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be588e-b40c-4bba-b3e6-42d42bd47991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:58.664355 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be588e-b40c-4bba-b3e6-42d42bd47991 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:58.664642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be588e-b40c-4bba-b3e6-42d42bd47991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:58.664913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be588e-b40c-4bba-b3e6-42d42bd47991 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:58.665685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be588e-b40c-4bba-b3e6-42d42bd47991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:58.667628 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-8edb3c66-957a-4775-9056-6be276161232 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:58.675047 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be588e-b40c-4bba-b3e6-42d42bd47991 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:58.677966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-8edb3c66-957a-4775-9056-6be276161232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:58.713223 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 862/1725] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:58.719267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be588e-b40c-4bba-b3e6-42d42bd47991 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:58.742163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be588e-b40c-4bba-b3e6-42d42bd47991 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:58.766349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 864/1726] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:58.767179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9ec87-dc00-4ef4-8e95-129b8b9be061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:58.768120 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9ec87-dc00-4ef4-8e95-129b8b9be061 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:58.768405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9ec87-dc00-4ef4-8e95-129b8b9be061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:58.768680 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a9ec87-dc00-4ef4-8e95-129b8b9be061 admin admin] PATH_INFO: `/v3/projects/eb5f36730e88434bb5a9ebf4b62835f2/users/a140fd966710466fab2b3321eb652571/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:58.779108 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25811f5c-b32c-47bb-96ee-f4d88264e249 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:58.779108 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25811f5c-b32c-47bb-96ee-f4d88264e249 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:58.782137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25811f5c-b32c-47bb-96ee-f4d88264e249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:58.828944 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9ec87-dc00-4ef4-8e95-129b8b9be061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb5f36730e88434bb5a9ebf4b62835f2, user_id=a140fd966710466fab2b3321eb652571, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:58.832278 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a9ec87-dc00-4ef4-8e95-129b8b9be061 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:58.836605 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-25811f5c-b32c-47bb-96ee-f4d88264e249 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:58.844620 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-25811f5c-b32c-47bb-96ee-f4d88264e249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:58.898451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 865/1727] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:58.906474 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06004983-110a-4656-81b1-7d51e8b1b9d5 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:58.907501 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06004983-110a-4656-81b1-7d51e8b1b9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:58.908275 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06004983-110a-4656-81b1-7d51e8b1b9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:59.017764 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 863/1728] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:10:58 2022] PUT /identity/v3/projects/eb5f36730e88434bb5a9ebf4b62835f2/users/a140fd966710466fab2b3321eb652571/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:59.017764 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9ec87-dc00-4ef4-8e95-129b8b9be061 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:59.157820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-06004983-110a-4656-81b1-7d51e8b1b9d5 None None] MFA Rules not processed for user `a4f499707998403d8f833eb783599466`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:59.160718 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-06004983-110a-4656-81b1-7d51e8b1b9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:59.330772 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:10:59.351024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a9ec87-dc00-4ef4-8e95-129b8b9be061 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:59.376591 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e115891-f1d2-4d5f-be41-fdce9c1e047d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:59.381265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e115891-f1d2-4d5f-be41-fdce9c1e047d admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:59.381940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e115891-f1d2-4d5f-be41-fdce9c1e047d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:59.384299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e115891-f1d2-4d5f-be41-fdce9c1e047d admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:59.384602 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 866/1729] 10.222.0.27 () {58 vars in 1009 bytes} [Sat Jul 16 15:10:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 486 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:59.385472 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06004983-110a-4656-81b1-7d51e8b1b9d5 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:59.386243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e115891-f1d2-4d5f-be41-fdce9c1e047d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:10:59.390422 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e115891-f1d2-4d5f-be41-fdce9c1e047d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:59.394439 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e115891-f1d2-4d5f-be41-fdce9c1e047d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:59.414639 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06004983-110a-4656-81b1-7d51e8b1b9d5 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:59.432991 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb929f86-f6c7-490b-bf47-e422d837e23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:59.433911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb929f86-f6c7-490b-bf47-e422d837e23e admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:59.434107 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb929f86-f6c7-490b-bf47-e422d837e23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:59.434389 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb929f86-f6c7-490b-bf47-e422d837e23e admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:59.435165 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb929f86-f6c7-490b-bf47-e422d837e23e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:59.436869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb929f86-f6c7-490b-bf47-e422d837e23e admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:59.467571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 864/1730] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:10:59.470945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e115891-f1d2-4d5f-be41-fdce9c1e047d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:59.476567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 867/1731] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:10:59.477878 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb929f86-f6c7-490b-bf47-e422d837e23e admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:59.500869 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb929f86-f6c7-490b-bf47-e422d837e23e admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:59.512990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ccc3c-b1d9-46e5-b966-f6e202998133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:59.513925 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ccc3c-b1d9-46e5-b966-f6e202998133 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:59.514210 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ccc3c-b1d9-46e5-b966-f6e202998133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:59.514534 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ccc3c-b1d9-46e5-b966-f6e202998133 admin admin] PATH_INFO: `/v3/projects/eb5f36730e88434bb5a9ebf4b62835f2/users/a140fd966710466fab2b3321eb652571/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:59.519940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ccc3c-b1d9-46e5-b966-f6e202998133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb5f36730e88434bb5a9ebf4b62835f2, user_id=a140fd966710466fab2b3321eb652571, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:59.520086 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e115891-f1d2-4d5f-be41-fdce9c1e047d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:10:59.523004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ccc3c-b1d9-46e5-b966-f6e202998133 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:59.546253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32786b3d-8a51-46e9-82cd-f1b90f762d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:10:59.548588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32786b3d-8a51-46e9-82cd-f1b90f762d3a admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:59.548900 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32786b3d-8a51-46e9-82cd-f1b90f762d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:59.552402 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32786b3d-8a51-46e9-82cd-f1b90f762d3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:59.554822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32786b3d-8a51-46e9-82cd-f1b90f762d3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:10:59.567195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 868/1732] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:10:59 2022] PUT /identity/v3/projects/eb5f36730e88434bb5a9ebf4b62835f2/users/a140fd966710466fab2b3321eb652571/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:10:59.567947 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32786b3d-8a51-46e9-82cd-f1b90f762d3a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:10:59.569679 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f446798-ba45-4c72-9c62-f904dbc86fb2 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:10:59.569979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f446798-ba45-4c72-9c62-f904dbc86fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:10:59.570874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f446798-ba45-4c72-9c62-f904dbc86fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:10:59.661584 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-0f446798-ba45-4c72-9c62-f904dbc86fb2 None None] MFA Rules not processed for user `a140fd966710466fab2b3321eb652571`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:10:59.666200 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-0f446798-ba45-4c72-9c62-f904dbc86fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:10:59.780182 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 865/1733] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:10:59 2022] POST /identity/v3/projects => generated 389 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:10:59.786400 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32786b3d-8a51-46e9-82cd-f1b90f762d3a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:59.919821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 869/1734] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:10:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:10:59.995224 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f446798-ba45-4c72-9c62-f904dbc86fb2 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:10:59.996504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-0f446798-ba45-4c72-9c62-f904dbc86fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:00.009649 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:00.146859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32786b3d-8a51-46e9-82cd-f1b90f762d3a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:00.172510 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2cfc7-10c1-4672-9efe-86379ebab7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:00.177236 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2cfc7-10c1-4672-9efe-86379ebab7d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:00.177817 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2cfc7-10c1-4672-9efe-86379ebab7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:00.178234 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2cfc7-10c1-4672-9efe-86379ebab7d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:00.181936 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b2cfc7-10c1-4672-9efe-86379ebab7d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:00.194164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b2cfc7-10c1-4672-9efe-86379ebab7d8 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:00.233008 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f446798-ba45-4c72-9c62-f904dbc86fb2 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:00.372586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 866/1735] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:10:59 2022] POST /identity/v3/users => generated 295 bytes in 571 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:11:00.374725 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2cfc7-10c1-4672-9efe-86379ebab7d8 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:00.388364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-07b2cfc7-10c1-4672-9efe-86379ebab7d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:00.491092 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fa546-c38b-4cdf-9d30-4309d63fe235 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1464c83fb294a3d90e5763ef648e834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:00.493520 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fa546-c38b-4cdf-9d30-4309d63fe235 service swift] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:00.495604 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fa546-c38b-4cdf-9d30-4309d63fe235 service swift] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:00.496686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fa546-c38b-4cdf-9d30-4309d63fe235 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:00.659558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95fa546-c38b-4cdf-9d30-4309d63fe235 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:00.659558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95fa546-c38b-4cdf-9d30-4309d63fe235 service swift] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:00.679340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2cfc7-10c1-4672-9efe-86379ebab7d8 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:00.685845 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99944e4c-3d1e-4d22-9bc8-2288a56362aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:00.688127 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99944e4c-3d1e-4d22-9bc8-2288a56362aa service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:00.688127 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99944e4c-3d1e-4d22-9bc8-2288a56362aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:00.688299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99944e4c-3d1e-4d22-9bc8-2288a56362aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:00.721657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 870/1736] 10.222.0.27 () {60 vars in 1384 bytes} [Sat Jul 16 15:10:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 727 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 15:11:00.723453 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fa546-c38b-4cdf-9d30-4309d63fe235 service swift] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:00.775452 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fa546-c38b-4cdf-9d30-4309d63fe235 service swift] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:00.798252 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc17f4-564f-48c0-95e1-70121caab817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:00.800323 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc17f4-564f-48c0-95e1-70121caab817 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:00.802701 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc17f4-564f-48c0-95e1-70121caab817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:00.803326 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc17f4-564f-48c0-95e1-70121caab817 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:00.807943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc17f4-564f-48c0-95e1-70121caab817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:00.812419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc17f4-564f-48c0-95e1-70121caab817 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:00.844160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 871/1737] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:00.852564 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc17f4-564f-48c0-95e1-70121caab817 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:00.913504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc17f4-564f-48c0-95e1-70121caab817 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:00.942346 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b354fef-603c-4802-8ae0-26de0b9f62c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:00.951166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b354fef-603c-4802-8ae0-26de0b9f62c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:00.951166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b354fef-603c-4802-8ae0-26de0b9f62c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:00.951166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b354fef-603c-4802-8ae0-26de0b9f62c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:00.954933 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b354fef-603c-4802-8ae0-26de0b9f62c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:00.959949 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b354fef-603c-4802-8ae0-26de0b9f62c1 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:01.028348 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 872/1738] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:01.037073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b354fef-603c-4802-8ae0-26de0b9f62c1 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:01.059847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b354fef-603c-4802-8ae0-26de0b9f62c1 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:01.085240 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56352120-c51f-4a7a-9d12-9cdcd3a293b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:01.088903 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56352120-c51f-4a7a-9d12-9cdcd3a293b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:01.091182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56352120-c51f-4a7a-9d12-9cdcd3a293b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:01.091929 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56352120-c51f-4a7a-9d12-9cdcd3a293b0 admin admin] PATH_INFO: `/v3/projects/021e2e0078f04c699d0f3ef3e200a204/users/f144ae9e21d84cd4935c07cd94335e9f/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:01.096758 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:01.141821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56352120-c51f-4a7a-9d12-9cdcd3a293b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=021e2e0078f04c699d0f3ef3e200a204, user_id=f144ae9e21d84cd4935c07cd94335e9f, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:01.152219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56352120-c51f-4a7a-9d12-9cdcd3a293b0 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:01.234079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99944e4c-3d1e-4d22-9bc8-2288a56362aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:01.250660 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99944e4c-3d1e-4d22-9bc8-2288a56362aa service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:01.261441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 873/1739] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:11:01 2022] PUT /identity/v3/projects/021e2e0078f04c699d0f3ef3e200a204/users/f144ae9e21d84cd4935c07cd94335e9f/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:01.287283 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422c8fc-9201-4fbc-b358-56accc4fd876 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:01.289646 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422c8fc-9201-4fbc-b358-56accc4fd876 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:01.290394 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422c8fc-9201-4fbc-b358-56accc4fd876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:01.365615 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 867/1740] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:00 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 984 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:11:01.454518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-8422c8fc-9201-4fbc-b358-56accc4fd876 None None] MFA Rules not processed for user `f144ae9e21d84cd4935c07cd94335e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:01.459888 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-8422c8fc-9201-4fbc-b358-56accc4fd876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:01.729135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99944e4c-3d1e-4d22-9bc8-2288a56362aa service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:01.741798 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99944e4c-3d1e-4d22-9bc8-2288a56362aa service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:01.757453 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:01.799859 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 874/1741] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 516 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:01.809377 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422c8fc-9201-4fbc-b358-56accc4fd876 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:01.961722 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d244016-70c0-4297-8159-23ddd20321df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:01.962553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d244016-70c0-4297-8159-23ddd20321df service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:01.963031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d244016-70c0-4297-8159-23ddd20321df service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:01.966053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d244016-70c0-4297-8159-23ddd20321df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:01.973254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5d244016-70c0-4297-8159-23ddd20321df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:02.227817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422c8fc-9201-4fbc-b358-56accc4fd876 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:02.232497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d244016-70c0-4297-8159-23ddd20321df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:02.237390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d244016-70c0-4297-8159-23ddd20321df service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:02.247806 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62cbbe-613f-4924-a76c-494aa2281d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:02.248855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62cbbe-613f-4924-a76c-494aa2281d8d admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:02.249292 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62cbbe-613f-4924-a76c-494aa2281d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:02.249717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62cbbe-613f-4924-a76c-494aa2281d8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:02.250646 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b62cbbe-613f-4924-a76c-494aa2281d8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:02.257039 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b62cbbe-613f-4924-a76c-494aa2281d8d admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:02.261523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 868/1742] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:11:02.264081 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354914b8-d489-4d4c-a300-3f3e97a9caca None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:02.264497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354914b8-d489-4d4c-a300-3f3e97a9caca None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:02.264879 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354914b8-d489-4d4c-a300-3f3e97a9caca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:02.316673 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-354914b8-d489-4d4c-a300-3f3e97a9caca None None] MFA Rules not processed for user `096fd1b2e2b4482da1026d3fb7e192c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:02.318869 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-354914b8-d489-4d4c-a300-3f3e97a9caca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:02.393556 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 875/1743] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:11:01 2022] POST /identity/v3/projects => generated 389 bytes in 586 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:11:02.397067 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62cbbe-613f-4924-a76c-494aa2281d8d admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:02.618400 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 869/1744] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:02.649331 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62cbbe-613f-4924-a76c-494aa2281d8d admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:02.658097 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9447d-7ed5-41df-8db1-dd942904cc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:02.659423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9447d-7ed5-41df-8db1-dd942904cc25 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:02.659758 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9447d-7ed5-41df-8db1-dd942904cc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:02.660396 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9447d-7ed5-41df-8db1-dd942904cc25 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:02.661710 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9447d-7ed5-41df-8db1-dd942904cc25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:02.667990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9447d-7ed5-41df-8db1-dd942904cc25 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:02.683377 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:02.803621 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 876/1745] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:11:02 2022] POST /identity/v3/users => generated 301 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:11:02.811557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354914b8-d489-4d4c-a300-3f3e97a9caca None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:02.867634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354914b8-d489-4d4c-a300-3f3e97a9caca None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:02.882687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea4807-be7c-4b66-b390-d05ead3b51e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:02.883619 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea4807-be7c-4b66-b390-d05ead3b51e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:02.883893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea4807-be7c-4b66-b390-d05ead3b51e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:02.884168 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea4807-be7c-4b66-b390-d05ead3b51e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:02.887862 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea4807-be7c-4b66-b390-d05ead3b51e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:02.890125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea4807-be7c-4b66-b390-d05ead3b51e5 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:02.911475 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 870/1746] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:02.917221 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9447d-7ed5-41df-8db1-dd942904cc25 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:02.951863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9447d-7ed5-41df-8db1-dd942904cc25 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:03.319138 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c510ad5-64bf-4473-a142-3b5b9da82556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:03.319138 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c510ad5-64bf-4473-a142-3b5b9da82556 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:03.319138 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c510ad5-64bf-4473-a142-3b5b9da82556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:03.319138 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c510ad5-64bf-4473-a142-3b5b9da82556 admin admin] PATH_INFO: `/v3/projects/890a2a7b49a24781b4aae0151680684c/users/d08cbddf52874bb797ac76ed6803388c/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:03.321011 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c510ad5-64bf-4473-a142-3b5b9da82556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=890a2a7b49a24781b4aae0151680684c, user_id=d08cbddf52874bb797ac76ed6803388c, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:03.321011 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c510ad5-64bf-4473-a142-3b5b9da82556 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:03.541450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 877/1747] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:11:02 2022] PUT /identity/v3/projects/890a2a7b49a24781b4aae0151680684c/users/d08cbddf52874bb797ac76ed6803388c/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:03.554204 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea4807-be7c-4b66-b390-d05ead3b51e5 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:03.662270 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea4807-be7c-4b66-b390-d05ead3b51e5 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:04.220129 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:04.292790 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b3cf4-eaf0-4f8d-8221-3f9c79151eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:04.314720 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b3cf4-eaf0-4f8d-8221-3f9c79151eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:04.319630 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b3cf4-eaf0-4f8d-8221-3f9c79151eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:04.319630 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b3cf4-eaf0-4f8d-8221-3f9c79151eb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:04.330939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b3cf4-eaf0-4f8d-8221-3f9c79151eb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:04.381982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b3cf4-eaf0-4f8d-8221-3f9c79151eb0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:04.486162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 871/1748] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 941 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:04.492910 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c510ad5-64bf-4473-a142-3b5b9da82556 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:04.546455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c510ad5-64bf-4473-a142-3b5b9da82556 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:04.605437 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8265e07-dfbf-48a0-b130-a14a8d245b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:04.616889 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8265e07-dfbf-48a0-b130-a14a8d245b09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:04.619735 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8265e07-dfbf-48a0-b130-a14a8d245b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:04.619735 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8265e07-dfbf-48a0-b130-a14a8d245b09 admin admin] PATH_INFO: `/v3/projects/890a2a7b49a24781b4aae0151680684c/users/d08cbddf52874bb797ac76ed6803388c/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:04.637632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8265e07-dfbf-48a0-b130-a14a8d245b09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=890a2a7b49a24781b4aae0151680684c, user_id=d08cbddf52874bb797ac76ed6803388c, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:04.644390 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8265e07-dfbf-48a0-b130-a14a8d245b09 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:04.787496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 878/1749] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:11:04 2022] PUT /identity/v3/projects/890a2a7b49a24781b4aae0151680684c/users/d08cbddf52874bb797ac76ed6803388c/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:04.808486 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a10f9-e8ee-4b71-8d1f-3a9e0063cdb5 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:04.809327 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a10f9-e8ee-4b71-8d1f-3a9e0063cdb5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:04.809327 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a10f9-e8ee-4b71-8d1f-3a9e0063cdb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:04.940274 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-0f7a10f9-e8ee-4b71-8d1f-3a9e0063cdb5 None None] MFA Rules not processed for user `d08cbddf52874bb797ac76ed6803388c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:04.977718 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-0f7a10f9-e8ee-4b71-8d1f-3a9e0063cdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:05.369636 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 872/1750] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 558 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:05.820775 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a36b4a-d879-4be8-9c46-3108268185a0 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:05.821366 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a36b4a-d879-4be8-9c46-3108268185a0 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:05.830205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a36b4a-d879-4be8-9c46-3108268185a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:05.901211 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-e3a36b4a-d879-4be8-9c46-3108268185a0 None None] MFA Rules not processed for user `6e372cc62af24664b01a967435eb8902`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:05.902672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e3a36b4a-d879-4be8-9c46-3108268185a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:06.071617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a10f9-e8ee-4b71-8d1f-3a9e0063cdb5 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:06.190195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 879/1751] 10.222.0.33 () {58 vars in 1015 bytes} [Sat Jul 16 15:11:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:06.254655 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a10f9-e8ee-4b71-8d1f-3a9e0063cdb5 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:06.262330 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a36b4a-d879-4be8-9c46-3108268185a0 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:06.268253 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e3a36b4a-d879-4be8-9c46-3108268185a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:06.275866 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c6766-725d-4ceb-9bb9-d0607af1cca7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:06.277886 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c6766-725d-4ceb-9bb9-d0607af1cca7 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:06.278192 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c6766-725d-4ceb-9bb9-d0607af1cca7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:06.278474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c6766-725d-4ceb-9bb9-d0607af1cca7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:06.318570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c6766-725d-4ceb-9bb9-d0607af1cca7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:06.322567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797c6766-725d-4ceb-9bb9-d0607af1cca7 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:06.334057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a36b4a-d879-4be8-9c46-3108268185a0 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:06.368802 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93ebe7-8b54-40ca-afb3-ce5d46b5b454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:06.370529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93ebe7-8b54-40ca-afb3-ce5d46b5b454 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:06.371135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93ebe7-8b54-40ca-afb3-ce5d46b5b454 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:06.378022 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93ebe7-8b54-40ca-afb3-ce5d46b5b454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:06.378478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 873/1752] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:11:06.442048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93ebe7-8b54-40ca-afb3-ce5d46b5b454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:06.461711 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93ebe7-8b54-40ca-afb3-ce5d46b5b454 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:06.557632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 880/1753] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:11:07.118374 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c6766-725d-4ceb-9bb9-d0607af1cca7 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:07.434441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797c6766-725d-4ceb-9bb9-d0607af1cca7 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:07.449158 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1327e95-7c51-4572-b3a1-d5de478b0b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:07.451649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1327e95-7c51-4572-b3a1-d5de478b0b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:07.451649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1327e95-7c51-4572-b3a1-d5de478b0b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:07.451649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1327e95-7c51-4572-b3a1-d5de478b0b31 admin admin] PATH_INFO: `/v3/users/f144ae9e21d84cd4935c07cd94335e9f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:07.456370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1327e95-7c51-4572-b3a1-d5de478b0b31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f144ae9e21d84cd4935c07cd94335e9f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:07.471012 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1327e95-7c51-4572-b3a1-d5de478b0b31 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:07.566792 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:07.854712 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b1327e95-7c51-4572-b3a1-d5de478b0b31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f144ae9e21d84cd4935c07cd94335e9f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:08.039509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b1327e95-7c51-4572-b3a1-d5de478b0b31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f144ae9e21d84cd4935c07cd94335e9f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:08.040194 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b1327e95-7c51-4572-b3a1-d5de478b0b31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f144ae9e21d84cd4935c07cd94335e9f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:08.055146 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b1327e95-7c51-4572-b3a1-d5de478b0b31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f144ae9e21d84cd4935c07cd94335e9f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:08.055827 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b1327e95-7c51-4572-b3a1-d5de478b0b31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f144ae9e21d84cd4935c07cd94335e9f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:08.131577 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 874/1754] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:11:07 2022] DELETE /identity/v3/users/f144ae9e21d84cd4935c07cd94335e9f => generated 0 bytes in 1014 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:08.141018 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85852f79-891e-429e-b980-8343649459de None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:08.141670 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85852f79-891e-429e-b980-8343649459de None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:08.142059 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85852f79-891e-429e-b980-8343649459de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:08.306812 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-85852f79-891e-429e-b980-8343649459de None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:08.319601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-85852f79-891e-429e-b980-8343649459de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:08.827668 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 881/1755] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 689 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:08.891250 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1327e95-7c51-4572-b3a1-d5de478b0b31 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:08.893826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b1327e95-7c51-4572-b3a1-d5de478b0b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:09.187269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1327e95-7c51-4572-b3a1-d5de478b0b31 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:09.197339 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0166e5-6fd1-453b-aa3c-7c204f4fabe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:09.200714 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0166e5-6fd1-453b-aa3c-7c204f4fabe2 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:09.201264 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0166e5-6fd1-453b-aa3c-7c204f4fabe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:09.202434 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0166e5-6fd1-453b-aa3c-7c204f4fabe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:09.232439 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0166e5-6fd1-453b-aa3c-7c204f4fabe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:09.235233 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0166e5-6fd1-453b-aa3c-7c204f4fabe2 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:09.252350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 875/1756] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:11:10.392174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85852f79-891e-429e-b980-8343649459de None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:10.397967 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-85852f79-891e-429e-b980-8343649459de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:10.457729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85852f79-891e-429e-b980-8343649459de None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:10.484799 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918466a2-5e37-485d-bfcc-66a0832e2ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:10.489714 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918466a2-5e37-485d-bfcc-66a0832e2ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:10.490000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918466a2-5e37-485d-bfcc-66a0832e2ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:10.490288 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918466a2-5e37-485d-bfcc-66a0832e2ea2 admin admin] PATH_INFO: `/v3/projects/021e2e0078f04c699d0f3ef3e200a204` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:10.498182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918466a2-5e37-485d-bfcc-66a0832e2ea2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=021e2e0078f04c699d0f3ef3e200a204)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:10.511045 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918466a2-5e37-485d-bfcc-66a0832e2ea2 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:10.810460 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:INSERT=5,keystone:SELECT=92,keystone:DELETE=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:10.828770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-918466a2-5e37-485d-bfcc-66a0832e2ea2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '021e2e0078f04c699d0f3ef3e200a204', 'request_id': 'req-918466a2-5e37-485d-bfcc-66a0832e2ea2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:10.845841 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-918466a2-5e37-485d-bfcc-66a0832e2ea2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '021e2e0078f04c699d0f3ef3e200a204', 'request_id': 'req-918466a2-5e37-485d-bfcc-66a0832e2ea2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:10.893082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-918466a2-5e37-485d-bfcc-66a0832e2ea2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '021e2e0078f04c699d0f3ef3e200a204', 'request_id': 'req-918466a2-5e37-485d-bfcc-66a0832e2ea2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:10.949917 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7fc46-2280-4ae5-96ce-60c0d382f996 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:10.951471 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7fc46-2280-4ae5-96ce-60c0d382f996 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:10.952372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7fc46-2280-4ae5-96ce-60c0d382f996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:11.015005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-918466a2-5e37-485d-bfcc-66a0832e2ea2 admin admin] The token cache is being invalidate because project 021e2e0078f04c699d0f3ef3e200a204 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:11:11.015005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-918466a2-5e37-485d-bfcc-66a0832e2ea2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:11.017926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-918466a2-5e37-485d-bfcc-66a0832e2ea2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:11.023450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 882/1757] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:11:10 2022] DELETE /identity/v3/projects/021e2e0078f04c699d0f3ef3e200a204 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:11.043164 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918466a2-5e37-485d-bfcc-66a0832e2ea2 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:11.048900 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-918466a2-5e37-485d-bfcc-66a0832e2ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:11.071910 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-87b7fc46-2280-4ae5-96ce-60c0d382f996 None None] MFA Rules not processed for user `9ee0106af0894acb911831433ca22caf`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:11.082006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-87b7fc46-2280-4ae5-96ce-60c0d382f996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:11.374331 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 876/1758] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:11.381523 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918466a2-5e37-485d-bfcc-66a0832e2ea2 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:11.415097 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d13d3-52ce-4b52-bf5b-2edaa7765f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:11.417665 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d13d3-52ce-4b52-bf5b-2edaa7765f0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:11.418443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d13d3-52ce-4b52-bf5b-2edaa7765f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:11.418728 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d13d3-52ce-4b52-bf5b-2edaa7765f0a admin admin] PATH_INFO: `/v3/users/d08cbddf52874bb797ac76ed6803388c` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:11.422822 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d13d3-52ce-4b52-bf5b-2edaa7765f0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d08cbddf52874bb797ac76ed6803388c)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:11.438146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d13d3-52ce-4b52-bf5b-2edaa7765f0a admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:11.758020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8333f7-490c-4e56-a5c5-b92bc090bde2 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:11.758558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8333f7-490c-4e56-a5c5-b92bc090bde2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:11.759006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8333f7-490c-4e56-a5c5-b92bc090bde2 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:11.765369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 877/1759] 10.222.0.27 () {54 vars in 921 bytes} [Sat Jul 16 15:11:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:11:11.816692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c966f-73b6-42c8-baeb-eb2ac9f6dbfb None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:11.818300 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c966f-73b6-42c8-baeb-eb2ac9f6dbfb None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:11.818828 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c966f-73b6-42c8-baeb-eb2ac9f6dbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:11.879339 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-600c966f-73b6-42c8-baeb-eb2ac9f6dbfb None None] MFA Rules not processed for user `3f2e241494244edf8d283e8f44192c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:11.886349 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-600c966f-73b6-42c8-baeb-eb2ac9f6dbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:11.890464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c38d13d3-52ce-4b52-bf5b-2edaa7765f0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd08cbddf52874bb797ac76ed6803388c'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:12.039609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c38d13d3-52ce-4b52-bf5b-2edaa7765f0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd08cbddf52874bb797ac76ed6803388c'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:12.055868 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c38d13d3-52ce-4b52-bf5b-2edaa7765f0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd08cbddf52874bb797ac76ed6803388c'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:12.116913 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c38d13d3-52ce-4b52-bf5b-2edaa7765f0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd08cbddf52874bb797ac76ed6803388c'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:12.116913 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c38d13d3-52ce-4b52-bf5b-2edaa7765f0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd08cbddf52874bb797ac76ed6803388c'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:12.117093 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:12.125475 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 883/1760] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:11:11 2022] DELETE /identity/v3/users/d08cbddf52874bb797ac76ed6803388c => generated 0 bytes in 1096 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:12.293688 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 878/1761] 10.222.0.27 () {58 vars in 1018 bytes} [Sat Jul 16 15:11:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:11:12.744438 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181bfe8-d1a1-4a4b-bbb1-ee2161152b3f None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:12.744438 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181bfe8-d1a1-4a4b-bbb1-ee2161152b3f None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:12.745656 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181bfe8-d1a1-4a4b-bbb1-ee2161152b3f None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:12.749394 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 884/1762] 10.222.0.27 () {54 vars in 921 bytes} [Sat Jul 16 15:11:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:11:13.271620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165735c-7f90-4a11-8ecf-6ee526f65ded None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:13.272627 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165735c-7f90-4a11-8ecf-6ee526f65ded None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:13.273270 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165735c-7f90-4a11-8ecf-6ee526f65ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:13.358984 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a165735c-7f90-4a11-8ecf-6ee526f65ded None None] MFA Rules not processed for user `3f2e241494244edf8d283e8f44192c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:13.362344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a165735c-7f90-4a11-8ecf-6ee526f65ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:13.633287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 879/1763] 10.222.0.27 () {58 vars in 1018 bytes} [Sat Jul 16 15:11:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:13.638391 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181bfe8-d1a1-4a4b-bbb1-ee2161152b3f None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:13.648148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3181bfe8-d1a1-4a4b-bbb1-ee2161152b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:13.730206 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181bfe8-d1a1-4a4b-bbb1-ee2161152b3f None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:13.772408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f3c18-78ff-4e3a-856e-9130d779a977 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:13.777524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f3c18-78ff-4e3a-856e-9130d779a977 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:13.778651 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f3c18-78ff-4e3a-856e-9130d779a977 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:13.779567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f3c18-78ff-4e3a-856e-9130d779a977 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:13.803047 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f3c18-78ff-4e3a-856e-9130d779a977 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:13.807464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f3c18-78ff-4e3a-856e-9130d779a977 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:13.834528 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 885/1764] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:11:14.176520 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165735c-7f90-4a11-8ecf-6ee526f65ded None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:14.181616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a165735c-7f90-4a11-8ecf-6ee526f65ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:14.238754 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165735c-7f90-4a11-8ecf-6ee526f65ded None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:14.265246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fcdea8-1681-4ec0-b3b0-8e246ac8d0ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:14.265956 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcdea8-1681-4ec0-b3b0-8e246ac8d0ae service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:14.266244 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcdea8-1681-4ec0-b3b0-8e246ac8d0ae service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:14.266530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcdea8-1681-4ec0-b3b0-8e246ac8d0ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:14.294006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fcdea8-1681-4ec0-b3b0-8e246ac8d0ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:14.300524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fcdea8-1681-4ec0-b3b0-8e246ac8d0ae service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:14.338850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 880/1765] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:11:14.523080 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f3c18-78ff-4e3a-856e-9130d779a977 service nova] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:14.524678 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-cf7f3c18-78ff-4e3a-856e-9130d779a977 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:14.647726 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:14.703536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f3c18-78ff-4e3a-856e-9130d779a977 service nova] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:14.711648 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6357b-56e9-459f-bf4e-08fc6a79c319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:14.712793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6357b-56e9-459f-bf4e-08fc6a79c319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:14.713076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6357b-56e9-459f-bf4e-08fc6a79c319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:14.713351 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6357b-56e9-459f-bf4e-08fc6a79c319 admin admin] PATH_INFO: `/v3/projects/890a2a7b49a24781b4aae0151680684c` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:14.715081 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6357b-56e9-459f-bf4e-08fc6a79c319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=890a2a7b49a24781b4aae0151680684c)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:14.717320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6357b-56e9-459f-bf4e-08fc6a79c319 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:14.921456 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5cd6357b-56e9-459f-bf4e-08fc6a79c319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '890a2a7b49a24781b4aae0151680684c', 'request_id': 'req-5cd6357b-56e9-459f-bf4e-08fc6a79c319'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:14.933490 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5cd6357b-56e9-459f-bf4e-08fc6a79c319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '890a2a7b49a24781b4aae0151680684c', 'request_id': 'req-5cd6357b-56e9-459f-bf4e-08fc6a79c319'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:14.953264 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5cd6357b-56e9-459f-bf4e-08fc6a79c319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '890a2a7b49a24781b4aae0151680684c', 'request_id': 'req-5cd6357b-56e9-459f-bf4e-08fc6a79c319'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:15.044787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5cd6357b-56e9-459f-bf4e-08fc6a79c319 admin admin] The token cache is being invalidate because project 890a2a7b49a24781b4aae0151680684c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:11:15.045129 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5cd6357b-56e9-459f-bf4e-08fc6a79c319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:15.046596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5cd6357b-56e9-459f-bf4e-08fc6a79c319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:15.047873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 886/1766] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:11:14 2022] DELETE /identity/v3/projects/890a2a7b49a24781b4aae0151680684c => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:11:15.054949 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc53f7-a16b-463f-8405-1057a160854c None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:15.056357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc53f7-a16b-463f-8405-1057a160854c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:15.056922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc53f7-a16b-463f-8405-1057a160854c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:15.085639 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-6efc53f7-a16b-463f-8405-1057a160854c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:15.086970 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6efc53f7-a16b-463f-8405-1057a160854c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:15.216241 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 881/1767] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:15.225799 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c372e7c-c53a-49f2-a343-b0fce375a877 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:15.226268 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c372e7c-c53a-49f2-a343-b0fce375a877 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:15.226673 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c372e7c-c53a-49f2-a343-b0fce375a877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:15.280497 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-9c372e7c-c53a-49f2-a343-b0fce375a877 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:15.281898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-9c372e7c-c53a-49f2-a343-b0fce375a877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:15.292851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 887/1768] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:15.299488 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc53f7-a16b-463f-8405-1057a160854c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:15.321158 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc53f7-a16b-463f-8405-1057a160854c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:15.329642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ca6ca-6305-4c7b-836e-3cd411b2b74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:15.330807 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ca6ca-6305-4c7b-836e-3cd411b2b74b admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:15.331934 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ca6ca-6305-4c7b-836e-3cd411b2b74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:15.333054 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ca6ca-6305-4c7b-836e-3cd411b2b74b admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:15.334015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ca6ca-6305-4c7b-836e-3cd411b2b74b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:11:15.338539 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ca6ca-6305-4c7b-836e-3cd411b2b74b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:15.346176 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ca6ca-6305-4c7b-836e-3cd411b2b74b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:15.383630 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 882/1769] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:11:15.390359 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c372e7c-c53a-49f2-a343-b0fce375a877 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:15.405543 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c372e7c-c53a-49f2-a343-b0fce375a877 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:15.439872 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb48ad-bdc3-4b14-b12c-a95752f6211e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:15.441648 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb48ad-bdc3-4b14-b12c-a95752f6211e admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:15.442130 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb48ad-bdc3-4b14-b12c-a95752f6211e admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:15.442608 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb48ad-bdc3-4b14-b12c-a95752f6211e admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:15.447678 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb48ad-bdc3-4b14-b12c-a95752f6211e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:15.451742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb48ad-bdc3-4b14-b12c-a95752f6211e admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:15.588370 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 888/1770] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:11:15 2022] POST /identity/v3/projects => generated 371 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:11:15.594658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ca6ca-6305-4c7b-836e-3cd411b2b74b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:15.615477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ca6ca-6305-4c7b-836e-3cd411b2b74b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:15.689269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:15.717986 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb46dfc-3cce-4a5a-a157-17ef75b86f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:15.718904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb46dfc-3cce-4a5a-a157-17ef75b86f59 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:15.719362 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb46dfc-3cce-4a5a-a157-17ef75b86f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:15.722774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb46dfc-3cce-4a5a-a157-17ef75b86f59 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:15.722774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb46dfc-3cce-4a5a-a157-17ef75b86f59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:15.724026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb46dfc-3cce-4a5a-a157-17ef75b86f59 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:15.821239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 883/1771] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:11:15 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:11:15.823711 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb48ad-bdc3-4b14-b12c-a95752f6211e admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:15.836990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb48ad-bdc3-4b14-b12c-a95752f6211e admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:15.843753 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffba0c1-1307-495e-beb1-59e031a684c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:15.846476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffba0c1-1307-495e-beb1-59e031a684c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:15.846476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffba0c1-1307-495e-beb1-59e031a684c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:15.846476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffba0c1-1307-495e-beb1-59e031a684c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:15.846476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffba0c1-1307-495e-beb1-59e031a684c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:15.849057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffba0c1-1307-495e-beb1-59e031a684c5 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:15.867537 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 889/1772] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:15.872352 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb46dfc-3cce-4a5a-a157-17ef75b86f59 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:15.887917 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb46dfc-3cce-4a5a-a157-17ef75b86f59 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:15.893554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb25d46-8c53-4981-820f-eaebd5a748d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:15.895237 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb25d46-8c53-4981-820f-eaebd5a748d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:15.895535 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb25d46-8c53-4981-820f-eaebd5a748d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:15.895813 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb25d46-8c53-4981-820f-eaebd5a748d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:15.898374 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb25d46-8c53-4981-820f-eaebd5a748d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:15.900203 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb25d46-8c53-4981-820f-eaebd5a748d5 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:15.914741 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 884/1773] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:15.920089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffba0c1-1307-495e-beb1-59e031a684c5 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:15.934444 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffba0c1-1307-495e-beb1-59e031a684c5 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:15.942534 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4819e70-d4c6-4585-a90e-54bcf36c515d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:15.944226 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4819e70-d4c6-4585-a90e-54bcf36c515d admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:15.944655 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4819e70-d4c6-4585-a90e-54bcf36c515d admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:15.945383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4819e70-d4c6-4585-a90e-54bcf36c515d admin admin] PATH_INFO: `/v3/projects/f5e8428fd1c849a1af745df37bf095b9/users/05da244187d54a9a8b5e4e623c3eee32/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:15.974983 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4819e70-d4c6-4585-a90e-54bcf36c515d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5e8428fd1c849a1af745df37bf095b9, user_id=05da244187d54a9a8b5e4e623c3eee32, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:15.980834 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4819e70-d4c6-4585-a90e-54bcf36c515d admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:16.049446 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 890/1774] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:11:15 2022] PUT /identity/v3/projects/f5e8428fd1c849a1af745df37bf095b9/users/05da244187d54a9a8b5e4e623c3eee32/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:16.062834 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b4bc9-1e4a-49c6-92e3-f799d1e761b7 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:16.063135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b4bc9-1e4a-49c6-92e3-f799d1e761b7 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:16.063429 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b4bc9-1e4a-49c6-92e3-f799d1e761b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:16.089507 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-4f0b4bc9-1e4a-49c6-92e3-f799d1e761b7 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:16.091200 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4f0b4bc9-1e4a-49c6-92e3-f799d1e761b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:16.176523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 885/1775] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:16.189787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4819e70-d4c6-4585-a90e-54bcf36c515d admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:16.194485 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b4819e70-d4c6-4585-a90e-54bcf36c515d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:16.291596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4819e70-d4c6-4585-a90e-54bcf36c515d admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:16.299245 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede3ed3-2104-4efb-9819-b739413edc43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:16.301809 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede3ed3-2104-4efb-9819-b739413edc43 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:16.302274 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede3ed3-2104-4efb-9819-b739413edc43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:16.302734 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede3ed3-2104-4efb-9819-b739413edc43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:16.322632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede3ed3-2104-4efb-9819-b739413edc43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:16.325167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede3ed3-2104-4efb-9819-b739413edc43 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:16.344764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 891/1776] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:11:19.178274 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48150491-1de3-4fc2-882c-159b157e9e27 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:19.179137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48150491-1de3-4fc2-882c-159b157e9e27 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:19.179137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48150491-1de3-4fc2-882c-159b157e9e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:19.227897 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-48150491-1de3-4fc2-882c-159b157e9e27 None None] MFA Rules not processed for user `05da244187d54a9a8b5e4e623c3eee32`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:19.230203 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-48150491-1de3-4fc2-882c-159b157e9e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:19.333219 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 886/1777] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:11:19.338567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede3ed3-2104-4efb-9819-b739413edc43 service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:19.363684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede3ed3-2104-4efb-9819-b739413edc43 service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:19.379064 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f527be-c23f-4433-bd7f-a13f603b1fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:19.380398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f527be-c23f-4433-bd7f-a13f603b1fad admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:19.382228 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f527be-c23f-4433-bd7f-a13f603b1fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:19.382811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f527be-c23f-4433-bd7f-a13f603b1fad admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:19.384567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f527be-c23f-4433-bd7f-a13f603b1fad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:19.390786 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f527be-c23f-4433-bd7f-a13f603b1fad admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:19.398583 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:19.548273 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 892/1778] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:11:19 2022] POST /identity/v3/projects => generated 373 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:11:19.550578 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48150491-1de3-4fc2-882c-159b157e9e27 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:19.565607 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48150491-1de3-4fc2-882c-159b157e9e27 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:19.694543 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:19.717520 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d5470-fdcd-418e-9bbf-fdfb0884ddd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:19.718818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d5470-fdcd-418e-9bbf-fdfb0884ddd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:19.719321 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d5470-fdcd-418e-9bbf-fdfb0884ddd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:19.719901 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d5470-fdcd-418e-9bbf-fdfb0884ddd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:19.721076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d5470-fdcd-418e-9bbf-fdfb0884ddd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:19.725022 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d5470-fdcd-418e-9bbf-fdfb0884ddd1 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:19.898510 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 887/1779] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:11:19 2022] POST /identity/v3/users => generated 293 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:11:19.903675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f527be-c23f-4433-bd7f-a13f603b1fad admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:19.916256 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f527be-c23f-4433-bd7f-a13f603b1fad admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:19.921912 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8548f5-21ca-4bdf-8cdb-3eb1de3320b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:19.922607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8548f5-21ca-4bdf-8cdb-3eb1de3320b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:19.922929 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8548f5-21ca-4bdf-8cdb-3eb1de3320b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:19.923239 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8548f5-21ca-4bdf-8cdb-3eb1de3320b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:19.926177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8548f5-21ca-4bdf-8cdb-3eb1de3320b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:19.926472 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8548f5-21ca-4bdf-8cdb-3eb1de3320b9 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:19.940907 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 893/1780] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:19.946443 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d5470-fdcd-418e-9bbf-fdfb0884ddd1 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:19.957497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d5470-fdcd-418e-9bbf-fdfb0884ddd1 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:19.964305 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8d8c5-3fa1-4716-87de-90de5a8ab111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:19.965815 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8d8c5-3fa1-4716-87de-90de5a8ab111 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:19.966410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8d8c5-3fa1-4716-87de-90de5a8ab111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:19.966983 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8d8c5-3fa1-4716-87de-90de5a8ab111 admin admin] PATH_INFO: `/v3/projects/1c06dcabce1a4ccbb6a2d1718ef43710/users/c6c879e740c740ea9091b7ab38c2d0f9/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:20.003567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8d8c5-3fa1-4716-87de-90de5a8ab111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c06dcabce1a4ccbb6a2d1718ef43710, user_id=c6c879e740c740ea9091b7ab38c2d0f9, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:20.010999 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8d8c5-3fa1-4716-87de-90de5a8ab111 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:20.129610 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 888/1781] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:11:19 2022] PUT /identity/v3/projects/1c06dcabce1a4ccbb6a2d1718ef43710/users/c6c879e740c740ea9091b7ab38c2d0f9/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:20.136477 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8548f5-21ca-4bdf-8cdb-3eb1de3320b9 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:20.150104 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8548f5-21ca-4bdf-8cdb-3eb1de3320b9 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:20.257173 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7e667-3b75-4292-8c71-e66f4dc668e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:20.258314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7e667-3b75-4292-8c71-e66f4dc668e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:20.258796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7e667-3b75-4292-8c71-e66f4dc668e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:20.259305 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7e667-3b75-4292-8c71-e66f4dc668e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:20.260642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7e667-3b75-4292-8c71-e66f4dc668e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:20.263763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7e667-3b75-4292-8c71-e66f4dc668e3 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:20.284546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 894/1782] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:20.289253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8d8c5-3fa1-4716-87de-90de5a8ab111 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:20.301169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8d8c5-3fa1-4716-87de-90de5a8ab111 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:20.306431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d57b9-0651-4d75-9b35-2a6b388c30dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:20.307418 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d57b9-0651-4d75-9b35-2a6b388c30dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:20.307703 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d57b9-0651-4d75-9b35-2a6b388c30dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:20.308120 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d57b9-0651-4d75-9b35-2a6b388c30dc admin admin] PATH_INFO: `/v3/projects/1c06dcabce1a4ccbb6a2d1718ef43710/users/c6c879e740c740ea9091b7ab38c2d0f9/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:20.310350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d57b9-0651-4d75-9b35-2a6b388c30dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c06dcabce1a4ccbb6a2d1718ef43710, user_id=c6c879e740c740ea9091b7ab38c2d0f9, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:20.313144 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d57b9-0651-4d75-9b35-2a6b388c30dc admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:20.327294 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 889/1783] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:11:20 2022] PUT /identity/v3/projects/1c06dcabce1a4ccbb6a2d1718ef43710/users/c6c879e740c740ea9091b7ab38c2d0f9/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:22.690990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7e667-3b75-4292-8c71-e66f4dc668e3 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:22.693749 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-41f7e667-3b75-4292-8c71-e66f4dc668e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:22.716855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7e667-3b75-4292-8c71-e66f4dc668e3 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:22.868917 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dcb465-0f14-42f4-8482-7a060eb85859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:22.868917 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcb465-0f14-42f4-8482-7a060eb85859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:22.868917 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcb465-0f14-42f4-8482-7a060eb85859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:22.868917 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcb465-0f14-42f4-8482-7a060eb85859 admin admin] PATH_INFO: `/v3/users/038884289cd74095bde16ab0f1d1834b` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:22.869704 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dcb465-0f14-42f4-8482-7a060eb85859 admin admin] RBAC: Authorizing `identity:delete_user(user_id=038884289cd74095bde16ab0f1d1834b)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:22.869704 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dcb465-0f14-42f4-8482-7a060eb85859 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:23.424349 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d57b9-0651-4d75-9b35-2a6b388c30dc admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:23.429366 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ba8d57b9-0651-4d75-9b35-2a6b388c30dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:23.450999 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d57b9-0651-4d75-9b35-2a6b388c30dc admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:23.458813 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d93370-87d2-4b4d-9bcd-dba40f8f61eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:23.460348 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d93370-87d2-4b4d-9bcd-dba40f8f61eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:23.461064 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d93370-87d2-4b4d-9bcd-dba40f8f61eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:23.461515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d93370-87d2-4b4d-9bcd-dba40f8f61eb admin admin] PATH_INFO: `/v3/users/81098bcea7884326b7f17258df7ea49b` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:23.464000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d93370-87d2-4b4d-9bcd-dba40f8f61eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=81098bcea7884326b7f17258df7ea49b)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:23.467426 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d93370-87d2-4b4d-9bcd-dba40f8f61eb admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:23.599891 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-70dcb465-0f14-42f4-8482-7a060eb85859 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '038884289cd74095bde16ab0f1d1834b'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:23.757504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-70dcb465-0f14-42f4-8482-7a060eb85859 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '038884289cd74095bde16ab0f1d1834b'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:23.789161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-70dcb465-0f14-42f4-8482-7a060eb85859 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '038884289cd74095bde16ab0f1d1834b'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:23.826561 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:23.850146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-70dcb465-0f14-42f4-8482-7a060eb85859 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '038884289cd74095bde16ab0f1d1834b'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:23.851575 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-70dcb465-0f14-42f4-8482-7a060eb85859 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '038884289cd74095bde16ab0f1d1834b'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:23.870220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 895/1784] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:11:22 2022] DELETE /identity/v3/users/038884289cd74095bde16ab0f1d1834b => generated 0 bytes in 1183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:11:23.877292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d7d93370-87d2-4b4d-9bcd-dba40f8f61eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81098bcea7884326b7f17258df7ea49b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:23.890940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d144f1-80c1-4fa8-b24d-39ca101d3098 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:23.891270 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d144f1-80c1-4fa8-b24d-39ca101d3098 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:23.891557 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d144f1-80c1-4fa8-b24d-39ca101d3098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:23.918020 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-87d144f1-80c1-4fa8-b24d-39ca101d3098 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:23.919525 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-87d144f1-80c1-4fa8-b24d-39ca101d3098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:24.050750 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 896/1785] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:24.063540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d144f1-80c1-4fa8-b24d-39ca101d3098 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:24.068518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-87d144f1-80c1-4fa8-b24d-39ca101d3098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:24.099717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d7d93370-87d2-4b4d-9bcd-dba40f8f61eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81098bcea7884326b7f17258df7ea49b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:24.100415 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d7d93370-87d2-4b4d-9bcd-dba40f8f61eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81098bcea7884326b7f17258df7ea49b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:24.101461 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d7d93370-87d2-4b4d-9bcd-dba40f8f61eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81098bcea7884326b7f17258df7ea49b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:24.113920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d7d93370-87d2-4b4d-9bcd-dba40f8f61eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81098bcea7884326b7f17258df7ea49b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:24.156990 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 890/1786] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:11:23 2022] DELETE /identity/v3/users/81098bcea7884326b7f17258df7ea49b => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:11:24.163626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd1578-32fc-42d7-b12b-abdf53767383 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:24.163851 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd1578-32fc-42d7-b12b-abdf53767383 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:24.164323 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd1578-32fc-42d7-b12b-abdf53767383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:24.199620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-00cd1578-32fc-42d7-b12b-abdf53767383 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:24.201025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-00cd1578-32fc-42d7-b12b-abdf53767383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:24.230388 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d144f1-80c1-4fa8-b24d-39ca101d3098 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:24.353544 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 891/1787] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:24.375219 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd1578-32fc-42d7-b12b-abdf53767383 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:24.382672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-00cd1578-32fc-42d7-b12b-abdf53767383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:24.407827 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d6b0d-9278-4d63-8fad-acba67988828 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:24.408841 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d6b0d-9278-4d63-8fad-acba67988828 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:24.409557 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d6b0d-9278-4d63-8fad-acba67988828 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:24.409557 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d6b0d-9278-4d63-8fad-acba67988828 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:24.411619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-4c9d6b0d-9278-4d63-8fad-acba67988828 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:24.422998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:24.428158 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d6b0d-9278-4d63-8fad-acba67988828 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:24.429819 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d6b0d-9278-4d63-8fad-acba67988828 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:24.448429 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 897/1788] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:11:24.450427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e026dd-755c-42d7-bc16-b50feac4f32c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:24.451025 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e026dd-755c-42d7-bc16-b50feac4f32c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:24.451592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e026dd-755c-42d7-bc16-b50feac4f32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:24.491063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-69e026dd-755c-42d7-bc16-b50feac4f32c None None] MFA Rules not processed for user `c6c879e740c740ea9091b7ab38c2d0f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:24.493120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-69e026dd-755c-42d7-bc16-b50feac4f32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:24.503751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd1578-32fc-42d7-b12b-abdf53767383 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:24.511353 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76e4cb-0da6-4268-ad5d-5837a3ad4d5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:24.512487 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76e4cb-0da6-4268-ad5d-5837a3ad4d5c service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:24.512980 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76e4cb-0da6-4268-ad5d-5837a3ad4d5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:24.513682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76e4cb-0da6-4268-ad5d-5837a3ad4d5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:24.540187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76e4cb-0da6-4268-ad5d-5837a3ad4d5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:24.543076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76e4cb-0da6-4268-ad5d-5837a3ad4d5c service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:24.575957 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 892/1789] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:11:24.638539 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 898/1790] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:24 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:24.644436 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76e4cb-0da6-4268-ad5d-5837a3ad4d5c service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:24.649296 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-cc76e4cb-0da6-4268-ad5d-5837a3ad4d5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:24.920281 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76e4cb-0da6-4268-ad5d-5837a3ad4d5c service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:24.927685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21aa7fb-b23e-47f3-88cf-d9919ff5b655 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:24.930015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21aa7fb-b23e-47f3-88cf-d9919ff5b655 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:24.930015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21aa7fb-b23e-47f3-88cf-d9919ff5b655 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:24.930278 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21aa7fb-b23e-47f3-88cf-d9919ff5b655 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:24.934014 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b21aa7fb-b23e-47f3-88cf-d9919ff5b655 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:25.072152 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21aa7fb-b23e-47f3-88cf-d9919ff5b655 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:25.075858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21aa7fb-b23e-47f3-88cf-d9919ff5b655 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:25.102873 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 893/1791] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:11:25.458657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e026dd-755c-42d7-bc16-b50feac4f32c None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:25.460277 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-69e026dd-755c-42d7-bc16-b50feac4f32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:25.477934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e026dd-755c-42d7-bc16-b50feac4f32c None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:25.496394 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fbd3d-5a2f-43c8-bd36-c38bbc5a6bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:25.496394 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fbd3d-5a2f-43c8-bd36-c38bbc5a6bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:25.496394 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fbd3d-5a2f-43c8-bd36-c38bbc5a6bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:25.496394 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fbd3d-5a2f-43c8-bd36-c38bbc5a6bdb admin admin] PATH_INFO: `/v3/projects/de343aaf089b426184e5972f73a46b3e` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:25.496889 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fbd3d-5a2f-43c8-bd36-c38bbc5a6bdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=de343aaf089b426184e5972f73a46b3e)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:25.496889 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fbd3d-5a2f-43c8-bd36-c38bbc5a6bdb admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:25.741818 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-fa8fbd3d-5a2f-43c8-bd36-c38bbc5a6bdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de343aaf089b426184e5972f73a46b3e', 'request_id': 'req-fa8fbd3d-5a2f-43c8-bd36-c38bbc5a6bdb'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:25.755515 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-fa8fbd3d-5a2f-43c8-bd36-c38bbc5a6bdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de343aaf089b426184e5972f73a46b3e', 'request_id': 'req-fa8fbd3d-5a2f-43c8-bd36-c38bbc5a6bdb'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:25.766420 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-fa8fbd3d-5a2f-43c8-bd36-c38bbc5a6bdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de343aaf089b426184e5972f73a46b3e', 'request_id': 'req-fa8fbd3d-5a2f-43c8-bd36-c38bbc5a6bdb'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:25.911353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-fa8fbd3d-5a2f-43c8-bd36-c38bbc5a6bdb admin admin] The token cache is being invalidate because project de343aaf089b426184e5972f73a46b3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:11:25.911353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-fa8fbd3d-5a2f-43c8-bd36-c38bbc5a6bdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:25.911353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-fa8fbd3d-5a2f-43c8-bd36-c38bbc5a6bdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:25.911353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 899/1792] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:11:25 2022] DELETE /identity/v3/projects/de343aaf089b426184e5972f73a46b3e => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:25.912142 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21aa7fb-b23e-47f3-88cf-d9919ff5b655 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:25.912142 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b21aa7fb-b23e-47f3-88cf-d9919ff5b655 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:25.914346 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fbd3d-5a2f-43c8-bd36-c38bbc5a6bdb admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:25.920744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-fa8fbd3d-5a2f-43c8-bd36-c38bbc5a6bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:25.993101 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21aa7fb-b23e-47f3-88cf-d9919ff5b655 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:26.120670 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa2f97-94fc-4052-a2d0-69f7375186de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:26.121800 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2f97-94fc-4052-a2d0-69f7375186de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:26.122046 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2f97-94fc-4052-a2d0-69f7375186de admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:26.122320 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2f97-94fc-4052-a2d0-69f7375186de admin admin] PATH_INFO: `/v3/projects/5401441f79af467a90cda0501aa0d9ca` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:26.124782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa2f97-94fc-4052-a2d0-69f7375186de admin admin] RBAC: Authorizing `identity:delete_project(project_id=5401441f79af467a90cda0501aa0d9ca)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:26.129650 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa2f97-94fc-4052-a2d0-69f7375186de admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:26.182044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fbd3d-5a2f-43c8-bd36-c38bbc5a6bdb admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:26.203766 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a196f6-24ed-482b-9a86-eae40d75acba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:26.206782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a196f6-24ed-482b-9a86-eae40d75acba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:26.207426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a196f6-24ed-482b-9a86-eae40d75acba admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:26.208026 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a196f6-24ed-482b-9a86-eae40d75acba admin admin] PATH_INFO: `/v3/users/a140fd966710466fab2b3321eb652571` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:26.231306 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a196f6-24ed-482b-9a86-eae40d75acba admin admin] RBAC: Authorizing `identity:delete_user(user_id=a140fd966710466fab2b3321eb652571)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:26.246292 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a196f6-24ed-482b-9a86-eae40d75acba admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:26.340606 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:26.378353 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3bfa2f97-94fc-4052-a2d0-69f7375186de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5401441f79af467a90cda0501aa0d9ca', 'request_id': 'req-3bfa2f97-94fc-4052-a2d0-69f7375186de'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:26.385658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3bfa2f97-94fc-4052-a2d0-69f7375186de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5401441f79af467a90cda0501aa0d9ca', 'request_id': 'req-3bfa2f97-94fc-4052-a2d0-69f7375186de'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:26.396102 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3bfa2f97-94fc-4052-a2d0-69f7375186de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5401441f79af467a90cda0501aa0d9ca', 'request_id': 'req-3bfa2f97-94fc-4052-a2d0-69f7375186de'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:26.530732 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3bfa2f97-94fc-4052-a2d0-69f7375186de admin admin] The token cache is being invalidate because project 5401441f79af467a90cda0501aa0d9ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:11:26.531215 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3bfa2f97-94fc-4052-a2d0-69f7375186de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:26.532392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3bfa2f97-94fc-4052-a2d0-69f7375186de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:26.539000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 894/1793] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:11:25 2022] DELETE /identity/v3/projects/5401441f79af467a90cda0501aa0d9ca => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:26.542178 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728809f-6fe5-4c21-add1-c9d924757301 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:26.542818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728809f-6fe5-4c21-add1-c9d924757301 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:26.544729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728809f-6fe5-4c21-add1-c9d924757301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:26.578429 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-46a196f6-24ed-482b-9a86-eae40d75acba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a140fd966710466fab2b3321eb652571'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:26.591366 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5728809f-6fe5-4c21-add1-c9d924757301 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:26.594655 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5728809f-6fe5-4c21-add1-c9d924757301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:26.671856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-46a196f6-24ed-482b-9a86-eae40d75acba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a140fd966710466fab2b3321eb652571'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:26.694304 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-46a196f6-24ed-482b-9a86-eae40d75acba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a140fd966710466fab2b3321eb652571'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:26.743104 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-46a196f6-24ed-482b-9a86-eae40d75acba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a140fd966710466fab2b3321eb652571'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:26.745247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-46a196f6-24ed-482b-9a86-eae40d75acba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a140fd966710466fab2b3321eb652571'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:26.759128 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 895/1794] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:11:26.769164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a248a-f18f-4947-bc1f-975faaec08d1 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:26.771104 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 900/1795] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:11:25 2022] DELETE /identity/v3/users/a140fd966710466fab2b3321eb652571 => generated 0 bytes in 857 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:26.773142 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a248a-f18f-4947-bc1f-975faaec08d1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:26.773609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a248a-f18f-4947-bc1f-975faaec08d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:26.804649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-425a248a-f18f-4947-bc1f-975faaec08d1 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:26.807418 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-425a248a-f18f-4947-bc1f-975faaec08d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:27.017190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 896/1796] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:27.021509 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a196f6-24ed-482b-9a86-eae40d75acba admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:27.034504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a196f6-24ed-482b-9a86-eae40d75acba admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:27.056371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629da270-baa3-4f26-ad98-6304c67b2671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:27.057074 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629da270-baa3-4f26-ad98-6304c67b2671 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:27.057363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629da270-baa3-4f26-ad98-6304c67b2671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:27.057642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629da270-baa3-4f26-ad98-6304c67b2671 admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:27.058410 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629da270-baa3-4f26-ad98-6304c67b2671 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:11:27.058745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629da270-baa3-4f26-ad98-6304c67b2671 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:27.060447 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629da270-baa3-4f26-ad98-6304c67b2671 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:27.071950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:27.095042 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 901/1797] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 15:11:27.102801 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a248a-f18f-4947-bc1f-975faaec08d1 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:27.145116 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a248a-f18f-4947-bc1f-975faaec08d1 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:27.168961 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2dbaa0-cf57-43bf-9ce4-542e164c1038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:27.170399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dbaa0-cf57-43bf-9ce4-542e164c1038 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:27.170868 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dbaa0-cf57-43bf-9ce4-542e164c1038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:27.171382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dbaa0-cf57-43bf-9ce4-542e164c1038 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:27.173010 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2dbaa0-cf57-43bf-9ce4-542e164c1038 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:27.177600 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2dbaa0-cf57-43bf-9ce4-542e164c1038 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:27.357859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 897/1798] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:11:27 2022] POST /identity/v3/projects => generated 405 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:11:27.358801 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629da270-baa3-4f26-ad98-6304c67b2671 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:27.379531 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629da270-baa3-4f26-ad98-6304c67b2671 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:27.454291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2dbaa0-cf57-43bf-9ce4-542e164c1038 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:27.461444 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-0b2dbaa0-cf57-43bf-9ce4-542e164c1038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:27.604998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f58e7b-3217-4b77-9edd-97efbe044951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:27.605819 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f58e7b-3217-4b77-9edd-97efbe044951 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:27.606154 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f58e7b-3217-4b77-9edd-97efbe044951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:27.607775 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f58e7b-3217-4b77-9edd-97efbe044951 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:27.611312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f58e7b-3217-4b77-9edd-97efbe044951 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:27.616312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f58e7b-3217-4b77-9edd-97efbe044951 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:27.675084 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2dbaa0-cf57-43bf-9ce4-542e164c1038 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:27.682683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5883b8-f8ce-4cc5-b739-0a47150a0f14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:27.684077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5883b8-f8ce-4cc5-b739-0a47150a0f14 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:27.684803 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5883b8-f8ce-4cc5-b739-0a47150a0f14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:27.686030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5883b8-f8ce-4cc5-b739-0a47150a0f14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:27.690521 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6b5883b8-f8ce-4cc5-b739-0a47150a0f14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:27.725671 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 902/1799] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:11:27 2022] POST /identity/v3/users => generated 303 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:11:27.732064 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f58e7b-3217-4b77-9edd-97efbe044951 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:27.755164 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f58e7b-3217-4b77-9edd-97efbe044951 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:27.772189 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5289063-9a94-4fa5-9a58-d8f740ae8b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:27.774004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5289063-9a94-4fa5-9a58-d8f740ae8b00 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:27.774551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5289063-9a94-4fa5-9a58-d8f740ae8b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:27.777137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5289063-9a94-4fa5-9a58-d8f740ae8b00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:27.778156 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5289063-9a94-4fa5-9a58-d8f740ae8b00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:27.784592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5289063-9a94-4fa5-9a58-d8f740ae8b00 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:27.803342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 903/1800] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:11:27.808129 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5289063-9a94-4fa5-9a58-d8f740ae8b00 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:27.827222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5289063-9a94-4fa5-9a58-d8f740ae8b00 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:27.837361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2bd94b-e3a7-4a8d-8f24-c16ce72a579c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:27.841970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2bd94b-e3a7-4a8d-8f24-c16ce72a579c admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:27.841970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2bd94b-e3a7-4a8d-8f24-c16ce72a579c admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:27.841970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2bd94b-e3a7-4a8d-8f24-c16ce72a579c admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:27.841970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2bd94b-e3a7-4a8d-8f24-c16ce72a579c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:27.842380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2bd94b-e3a7-4a8d-8f24-c16ce72a579c admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:27.860833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 904/1801] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:27.866715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5883b8-f8ce-4cc5-b739-0a47150a0f14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:27.868533 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5883b8-f8ce-4cc5-b739-0a47150a0f14 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:27.875461 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2bd94b-e3a7-4a8d-8f24-c16ce72a579c admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:27.896213 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 898/1802] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:11:27.922611 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2bd94b-e3a7-4a8d-8f24-c16ce72a579c admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:27.955865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f06fe-f4bd-46fa-b08b-9b0bf289294a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:27.957486 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f06fe-f4bd-46fa-b08b-9b0bf289294a admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:27.957978 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f06fe-f4bd-46fa-b08b-9b0bf289294a admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:27.958432 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f06fe-f4bd-46fa-b08b-9b0bf289294a admin admin] PATH_INFO: `/v3/projects/c2a9fd6b8a024d37919a3ed2138e8174/users/c0386f74c7cb4795a2d6d7fed281dce2/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:28.038920 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f06fe-f4bd-46fa-b08b-9b0bf289294a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2a9fd6b8a024d37919a3ed2138e8174, user_id=c0386f74c7cb4795a2d6d7fed281dce2, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:28.048121 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f06fe-f4bd-46fa-b08b-9b0bf289294a admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:28.245140 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:29.030866 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5883b8-f8ce-4cc5-b739-0a47150a0f14 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:29.034415 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6b5883b8-f8ce-4cc5-b739-0a47150a0f14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:29.035135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 905/1803] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:11:27 2022] PUT /identity/v3/projects/c2a9fd6b8a024d37919a3ed2138e8174/users/c0386f74c7cb4795a2d6d7fed281dce2/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 1153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:29.059267 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fac24a-8c48-4a58-b3d8-f1c1be9bc4f2 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:29.059563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fac24a-8c48-4a58-b3d8-f1c1be9bc4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:29.059845 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fac24a-8c48-4a58-b3d8-f1c1be9bc4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:29.075829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5883b8-f8ce-4cc5-b739-0a47150a0f14 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:29.120245 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-b7fac24a-8c48-4a58-b3d8-f1c1be9bc4f2 None None] MFA Rules not processed for user `c0386f74c7cb4795a2d6d7fed281dce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:29.133343 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b7fac24a-8c48-4a58-b3d8-f1c1be9bc4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:29.272899 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:29.417212 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ce4d8-a52a-44b6-898d-e1c727143b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:29.417212 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ce4d8-a52a-44b6-898d-e1c727143b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:29.419497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ce4d8-a52a-44b6-898d-e1c727143b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:29.419947 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ce4d8-a52a-44b6-898d-e1c727143b16 admin admin] PATH_INFO: `/v3/projects/eb5f36730e88434bb5a9ebf4b62835f2` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:29.421576 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ce4d8-a52a-44b6-898d-e1c727143b16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb5f36730e88434bb5a9ebf4b62835f2)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:29.427431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ce4d8-a52a-44b6-898d-e1c727143b16 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:29.495562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 906/1804] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:29 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:29.510756 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fac24a-8c48-4a58-b3d8-f1c1be9bc4f2 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:29.613054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fac24a-8c48-4a58-b3d8-f1c1be9bc4f2 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:29.640671 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2365426-acf2-4df8-b352-c394af32f7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:29.644471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2365426-acf2-4df8-b352-c394af32f7b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:29.644471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2365426-acf2-4df8-b352-c394af32f7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:29.644892 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2365426-acf2-4df8-b352-c394af32f7b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:29.649659 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2365426-acf2-4df8-b352-c394af32f7b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:29.662074 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2365426-acf2-4df8-b352-c394af32f7b4 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:29.691312 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a06ce4d8-a52a-44b6-898d-e1c727143b16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb5f36730e88434bb5a9ebf4b62835f2', 'request_id': 'req-a06ce4d8-a52a-44b6-898d-e1c727143b16'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:29.709686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a06ce4d8-a52a-44b6-898d-e1c727143b16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb5f36730e88434bb5a9ebf4b62835f2', 'request_id': 'req-a06ce4d8-a52a-44b6-898d-e1c727143b16'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:29.725959 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a06ce4d8-a52a-44b6-898d-e1c727143b16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb5f36730e88434bb5a9ebf4b62835f2', 'request_id': 'req-a06ce4d8-a52a-44b6-898d-e1c727143b16'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:29.918614 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a06ce4d8-a52a-44b6-898d-e1c727143b16 admin admin] The token cache is being invalidate because project eb5f36730e88434bb5a9ebf4b62835f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:11:29.921055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a06ce4d8-a52a-44b6-898d-e1c727143b16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:29.922551 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a06ce4d8-a52a-44b6-898d-e1c727143b16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:29.930032 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 899/1805] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:11:29 2022] DELETE /identity/v3/projects/eb5f36730e88434bb5a9ebf4b62835f2 => generated 0 bytes in 904 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:29.961628 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec3cd1-a0b8-49d1-ba62-072e241e7abf None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:29.965094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec3cd1-a0b8-49d1-ba62-072e241e7abf None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:29.975658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec3cd1-a0b8-49d1-ba62-072e241e7abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:29.992323 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 907/1806] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:11:29 2022] POST /identity/v3/projects => generated 405 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:11:30.000274 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2365426-acf2-4df8-b352-c394af32f7b4 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:30.008868 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-f2365426-acf2-4df8-b352-c394af32f7b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:30.070559 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-92ec3cd1-a0b8-49d1-ba62-072e241e7abf None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:30.073464 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-92ec3cd1-a0b8-49d1-ba62-072e241e7abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:30.335754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2365426-acf2-4df8-b352-c394af32f7b4 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:30.350673 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3ada1-d7fd-4e18-ae8f-b036fd514c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:30.351469 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3ada1-d7fd-4e18-ae8f-b036fd514c95 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:30.351749 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3ada1-d7fd-4e18-ae8f-b036fd514c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:30.352025 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3ada1-d7fd-4e18-ae8f-b036fd514c95 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:30.352760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3ada1-d7fd-4e18-ae8f-b036fd514c95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:30.359700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3ada1-d7fd-4e18-ae8f-b036fd514c95 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:30.425866 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 900/1807] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 482 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:30.435438 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406d061-d21e-4a53-9973-d597ff183504 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:30.435730 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406d061-d21e-4a53-9973-d597ff183504 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:30.436094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406d061-d21e-4a53-9973-d597ff183504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:30.477781 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 908/1808] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:11:29 2022] POST /identity/v3/users => generated 309 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:11:30.483329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3ada1-d7fd-4e18-ae8f-b036fd514c95 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:30.490632 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-6406d061-d21e-4a53-9973-d597ff183504 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:30.494655 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6406d061-d21e-4a53-9973-d597ff183504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:30.507100 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3ada1-d7fd-4e18-ae8f-b036fd514c95 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:30.519145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 901/1809] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:11:30.522042 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6e1b2-153c-41e8-a6bc-ed2e8f1ee8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:30.523047 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6e1b2-153c-41e8-a6bc-ed2e8f1ee8c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:30.526138 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6e1b2-153c-41e8-a6bc-ed2e8f1ee8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:30.526617 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6e1b2-153c-41e8-a6bc-ed2e8f1ee8c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:30.527676 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6e1b2-153c-41e8-a6bc-ed2e8f1ee8c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:30.527804 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406d061-d21e-4a53-9973-d597ff183504 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:30.530183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6e1b2-153c-41e8-a6bc-ed2e8f1ee8c7 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:30.545238 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 909/1810] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:30.551304 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6e1b2-153c-41e8-a6bc-ed2e8f1ee8c7 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:30.557707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406d061-d21e-4a53-9973-d597ff183504 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:30.571792 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89934c-19d8-43a9-b27a-53dcdfb8bff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:30.573078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89934c-19d8-43a9-b27a-53dcdfb8bff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:30.573607 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89934c-19d8-43a9-b27a-53dcdfb8bff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:30.574247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89934c-19d8-43a9-b27a-53dcdfb8bff0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:30.574802 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6e1b2-153c-41e8-a6bc-ed2e8f1ee8c7 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:30.575728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89934c-19d8-43a9-b27a-53dcdfb8bff0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:11:30.576287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89934c-19d8-43a9-b27a-53dcdfb8bff0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:30.579845 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89934c-19d8-43a9-b27a-53dcdfb8bff0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:30.592870 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d4bce-1e7f-4a35-898b-5af5a63ce656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:30.594302 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d4bce-1e7f-4a35-898b-5af5a63ce656 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:30.595143 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d4bce-1e7f-4a35-898b-5af5a63ce656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:30.595630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d4bce-1e7f-4a35-898b-5af5a63ce656 admin admin] PATH_INFO: `/v3/projects/b68e8232db774f76b002decbf0f34361/users/de2311da1bf74dcba57c946e3342fe34/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:30.627987 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 902/1811] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:11:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 15:11:30.635789 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89934c-19d8-43a9-b27a-53dcdfb8bff0 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:30.641051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123d4bce-1e7f-4a35-898b-5af5a63ce656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b68e8232db774f76b002decbf0f34361, user_id=de2311da1bf74dcba57c946e3342fe34, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:30.649932 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123d4bce-1e7f-4a35-898b-5af5a63ce656 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:30.670521 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89934c-19d8-43a9-b27a-53dcdfb8bff0 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:30.688136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f95ac5-fc79-4138-b230-5291640910da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:30.689227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f95ac5-fc79-4138-b230-5291640910da admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:30.691582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f95ac5-fc79-4138-b230-5291640910da admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:30.692195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f95ac5-fc79-4138-b230-5291640910da admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:30.693589 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f95ac5-fc79-4138-b230-5291640910da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:30.697971 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f95ac5-fc79-4138-b230-5291640910da admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:30.759683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 910/1812] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:11:30 2022] PUT /identity/v3/projects/b68e8232db774f76b002decbf0f34361/users/de2311da1bf74dcba57c946e3342fe34/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:30.764901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d4bce-1e7f-4a35-898b-5af5a63ce656 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:30.827078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 903/1813] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:11:30 2022] POST /identity/v3/projects => generated 365 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:11:30.834209 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f95ac5-fc79-4138-b230-5291640910da admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:30.864423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d4bce-1e7f-4a35-898b-5af5a63ce656 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:30.973198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4eabb7-732e-41f2-9d8a-74387ae41b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:30.974055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4eabb7-732e-41f2-9d8a-74387ae41b53 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:30.974802 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4eabb7-732e-41f2-9d8a-74387ae41b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:30.975049 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4eabb7-732e-41f2-9d8a-74387ae41b53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:30.979049 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4eabb7-732e-41f2-9d8a-74387ae41b53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:30.982173 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4eabb7-732e-41f2-9d8a-74387ae41b53 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:31.004194 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f95ac5-fc79-4138-b230-5291640910da admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:31.011657 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e719a3-1a8c-4f78-a9c8-3f4548eb1603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:31.013705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e719a3-1a8c-4f78-a9c8-3f4548eb1603 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:31.014029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e719a3-1a8c-4f78-a9c8-3f4548eb1603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:31.014429 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e719a3-1a8c-4f78-a9c8-3f4548eb1603 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:31.015265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e719a3-1a8c-4f78-a9c8-3f4548eb1603 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:31.017793 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e719a3-1a8c-4f78-a9c8-3f4548eb1603 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:31.030873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 911/1814] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:11:31.046276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4eabb7-732e-41f2-9d8a-74387ae41b53 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:31.073271 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:31.081809 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4eabb7-732e-41f2-9d8a-74387ae41b53 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:31.095425 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6cc904-c663-4a9f-a6d0-1a8509514d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:31.099239 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cc904-c663-4a9f-a6d0-1a8509514d17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:31.099876 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cc904-c663-4a9f-a6d0-1a8509514d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:31.100443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cc904-c663-4a9f-a6d0-1a8509514d17 admin admin] PATH_INFO: `/v3/projects/b68e8232db774f76b002decbf0f34361/users/de2311da1bf74dcba57c946e3342fe34/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:31.107317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6cc904-c663-4a9f-a6d0-1a8509514d17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b68e8232db774f76b002decbf0f34361, user_id=de2311da1bf74dcba57c946e3342fe34, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:31.124315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6cc904-c663-4a9f-a6d0-1a8509514d17 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:31.151684 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 904/1815] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:11:30 2022] POST /identity/v3/users => generated 283 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:11:31.157665 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e719a3-1a8c-4f78-a9c8-3f4548eb1603 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:31.176209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:31.182997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 912/1816] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:11:31 2022] PUT /identity/v3/projects/b68e8232db774f76b002decbf0f34361/users/de2311da1bf74dcba57c946e3342fe34/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:31.200537 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82b5b8-09f6-406a-9894-5619146f3922 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:31.200537 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82b5b8-09f6-406a-9894-5619146f3922 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:31.200537 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82b5b8-09f6-406a-9894-5619146f3922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:31.207334 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e719a3-1a8c-4f78-a9c8-3f4548eb1603 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:31.279955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-0f82b5b8-09f6-406a-9894-5619146f3922 None None] MFA Rules not processed for user `de2311da1bf74dcba57c946e3342fe34`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:31.284661 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-0f82b5b8-09f6-406a-9894-5619146f3922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:31.320477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94d0d2-4f04-4f63-927d-c161c02506da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:31.321649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94d0d2-4f04-4f63-927d-c161c02506da admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:31.322137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94d0d2-4f04-4f63-927d-c161c02506da admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:31.323453 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94d0d2-4f04-4f63-927d-c161c02506da admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:31.323928 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94d0d2-4f04-4f63-927d-c161c02506da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:31.326978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94d0d2-4f04-4f63-927d-c161c02506da admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:31.381935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 905/1817] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:31.383654 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94d0d2-4f04-4f63-927d-c161c02506da admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:31.449519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94d0d2-4f04-4f63-927d-c161c02506da admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:31.480136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd671d8a-8ae2-426b-a42f-93ffac31c459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:31.481487 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd671d8a-8ae2-426b-a42f-93ffac31c459 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:31.481992 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd671d8a-8ae2-426b-a42f-93ffac31c459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:31.482622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd671d8a-8ae2-426b-a42f-93ffac31c459 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:31.497328 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd671d8a-8ae2-426b-a42f-93ffac31c459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:31.503567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd671d8a-8ae2-426b-a42f-93ffac31c459 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:31.520581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 913/1818] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:31 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:31.551216 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 906/1819] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:31.552783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f82b5b8-09f6-406a-9894-5619146f3922 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:31.576534 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f82b5b8-09f6-406a-9894-5619146f3922 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:31.626331 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb6536-a475-49fc-9271-71c384f9b98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:31.631146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb6536-a475-49fc-9271-71c384f9b98c admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:31.631146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb6536-a475-49fc-9271-71c384f9b98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:31.631146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb6536-a475-49fc-9271-71c384f9b98c admin admin] PATH_INFO: `/v3/projects/a18f9103b143427c8c2d52cfc3d19688/users/e111ab406c5140cd9d1bba56832c1ab2/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:31.669755 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd671d8a-8ae2-426b-a42f-93ffac31c459 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:31.674613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-cd671d8a-8ae2-426b-a42f-93ffac31c459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:31.680183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb6536-a475-49fc-9271-71c384f9b98c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a18f9103b143427c8c2d52cfc3d19688, user_id=e111ab406c5140cd9d1bba56832c1ab2, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:31.683440 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb6536-a475-49fc-9271-71c384f9b98c admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:31.821131 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 914/1820] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:11:31 2022] PUT /identity/v3/projects/a18f9103b143427c8c2d52cfc3d19688/users/e111ab406c5140cd9d1bba56832c1ab2/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:11:31.836656 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d42356-bc4b-4ba6-8769-c2cd39161449 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:31.837236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d42356-bc4b-4ba6-8769-c2cd39161449 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:31.837962 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d42356-bc4b-4ba6-8769-c2cd39161449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:31.936893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd671d8a-8ae2-426b-a42f-93ffac31c459 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:31.980836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-72d42356-bc4b-4ba6-8769-c2cd39161449 None None] MFA Rules not processed for user `e111ab406c5140cd9d1bba56832c1ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:31.982676 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-72d42356-bc4b-4ba6-8769-c2cd39161449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:32.086387 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:32.127387 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 915/1821] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:31 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:11:32.127774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee488e29-6845-4bcc-8e87-e1535d2c2509 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:32.127774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee488e29-6845-4bcc-8e87-e1535d2c2509 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:32.127774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee488e29-6845-4bcc-8e87-e1535d2c2509 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:32.127774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee488e29-6845-4bcc-8e87-e1535d2c2509 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:32.131995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d42356-bc4b-4ba6-8769-c2cd39161449 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:32.266701 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee488e29-6845-4bcc-8e87-e1535d2c2509 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:32.270832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee488e29-6845-4bcc-8e87-e1535d2c2509 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:32.294329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d42356-bc4b-4ba6-8769-c2cd39161449 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:32.302356 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2289a4-56fb-47f1-b645-2835cac3d2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:32.302657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2289a4-56fb-47f1-b645-2835cac3d2ed admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:32.302976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2289a4-56fb-47f1-b645-2835cac3d2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:32.303300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2289a4-56fb-47f1-b645-2835cac3d2ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:32.304027 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2289a4-56fb-47f1-b645-2835cac3d2ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:32.306228 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2289a4-56fb-47f1-b645-2835cac3d2ed admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:32.311308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 907/1822] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:31 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:11:32.559706 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 916/1823] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:11:32 2022] POST /identity/v3/projects => generated 365 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:11:32.568247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee488e29-6845-4bcc-8e87-e1535d2c2509 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:32.620784 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee488e29-6845-4bcc-8e87-e1535d2c2509 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:32.695912 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:32.775199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2289a4-56fb-47f1-b645-2835cac3d2ed admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:32.843109 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381ba4a-1cb1-491c-bf5c-6c21a14c0eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:32.844087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381ba4a-1cb1-491c-bf5c-6c21a14c0eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:32.844591 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381ba4a-1cb1-491c-bf5c-6c21a14c0eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:32.845144 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381ba4a-1cb1-491c-bf5c-6c21a14c0eb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:32.846624 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381ba4a-1cb1-491c-bf5c-6c21a14c0eb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:32.854019 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381ba4a-1cb1-491c-bf5c-6c21a14c0eb8 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:33.008512 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2289a4-56fb-47f1-b645-2835cac3d2ed admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:33.017865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e909cd-7f83-400c-9858-d4faf183e07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:33.019582 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e909cd-7f83-400c-9858-d4faf183e07b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:33.020138 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e909cd-7f83-400c-9858-d4faf183e07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:33.020917 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e909cd-7f83-400c-9858-d4faf183e07b admin admin] PATH_INFO: `/v3/users/c0386f74c7cb4795a2d6d7fed281dce2` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:33.025173 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e909cd-7f83-400c-9858-d4faf183e07b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0386f74c7cb4795a2d6d7fed281dce2)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:33.029220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e909cd-7f83-400c-9858-d4faf183e07b admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:33.082051 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 908/1824] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:11:32 2022] POST /identity/v3/users => generated 289 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:11:33.086705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381ba4a-1cb1-491c-bf5c-6c21a14c0eb8 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:33.102032 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381ba4a-1cb1-491c-bf5c-6c21a14c0eb8 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:33.110983 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85717940-540b-480f-b6d5-58cf20b6dbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:33.112886 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85717940-540b-480f-b6d5-58cf20b6dbdc admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:33.113594 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85717940-540b-480f-b6d5-58cf20b6dbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:33.115341 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85717940-540b-480f-b6d5-58cf20b6dbdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:33.118004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85717940-540b-480f-b6d5-58cf20b6dbdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:33.121461 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85717940-540b-480f-b6d5-58cf20b6dbdc admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:33.144308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 909/1825] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:33.147787 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85717940-540b-480f-b6d5-58cf20b6dbdc admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:33.164489 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85717940-540b-480f-b6d5-58cf20b6dbdc admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:33.179044 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41614656-345d-4e9f-a8ff-c940003a3dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:33.182362 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41614656-345d-4e9f-a8ff-c940003a3dee admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:33.183653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41614656-345d-4e9f-a8ff-c940003a3dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:33.187956 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41614656-345d-4e9f-a8ff-c940003a3dee admin admin] PATH_INFO: `/v3/projects/9d645d5b2f5c4e2baa5b1a6d214a95b9/users/cf60bf4e17934b2fa3aa409073982dcd/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:33.272704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41614656-345d-4e9f-a8ff-c940003a3dee admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d645d5b2f5c4e2baa5b1a6d214a95b9, user_id=cf60bf4e17934b2fa3aa409073982dcd, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:33.279721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41614656-345d-4e9f-a8ff-c940003a3dee admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:33.394821 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 910/1826] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:11:33 2022] PUT /identity/v3/projects/9d645d5b2f5c4e2baa5b1a6d214a95b9/users/cf60bf4e17934b2fa3aa409073982dcd/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:33.398824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a2e909cd-7f83-400c-9858-d4faf183e07b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0386f74c7cb4795a2d6d7fed281dce2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:33.400534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41614656-345d-4e9f-a8ff-c940003a3dee admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:33.493720 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a2e909cd-7f83-400c-9858-d4faf183e07b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0386f74c7cb4795a2d6d7fed281dce2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:33.501893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:33.509639 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a2e909cd-7f83-400c-9858-d4faf183e07b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0386f74c7cb4795a2d6d7fed281dce2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:33.608013 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a2e909cd-7f83-400c-9858-d4faf183e07b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0386f74c7cb4795a2d6d7fed281dce2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:33.608013 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a2e909cd-7f83-400c-9858-d4faf183e07b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0386f74c7cb4795a2d6d7fed281dce2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:33.636768 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41614656-345d-4e9f-a8ff-c940003a3dee admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:33.638787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 917/1827] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:11:32 2022] DELETE /identity/v3/users/c0386f74c7cb4795a2d6d7fed281dce2 => generated 0 bytes in 861 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:33.646424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbce966-85cb-40f3-8839-cc3603099a1e None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:33.649912 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbce966-85cb-40f3-8839-cc3603099a1e None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:33.650243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbce966-85cb-40f3-8839-cc3603099a1e None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:33.656387 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 918/1828] 10.222.0.27 () {54 vars in 921 bytes} [Sat Jul 16 15:11:33 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:11:33.664067 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e170bb8-5241-40a7-9822-c3b46fe6099c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:33.664362 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e170bb8-5241-40a7-9822-c3b46fe6099c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:33.664632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e170bb8-5241-40a7-9822-c3b46fe6099c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:33.721341 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:33.761158 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-6e170bb8-5241-40a7-9822-c3b46fe6099c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:33.765927 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-6e170bb8-5241-40a7-9822-c3b46fe6099c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:33.836059 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537cf375-01f2-49b9-b92f-f8ba8902ca48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:33.836683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cf375-01f2-49b9-b92f-f8ba8902ca48 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:33.836969 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cf375-01f2-49b9-b92f-f8ba8902ca48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:33.837608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cf375-01f2-49b9-b92f-f8ba8902ca48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:33.838018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537cf375-01f2-49b9-b92f-f8ba8902ca48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:33.839908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537cf375-01f2-49b9-b92f-f8ba8902ca48 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:33.858795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 911/1829] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:33.858795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadfdee-1f1e-4628-81b3-6dce731a4f25 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:33.858795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadfdee-1f1e-4628-81b3-6dce731a4f25 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:33.858795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadfdee-1f1e-4628-81b3-6dce731a4f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:33.887649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-efadfdee-1f1e-4628-81b3-6dce731a4f25 None None] MFA Rules not processed for user `3f2e241494244edf8d283e8f44192c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:33.889056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-efadfdee-1f1e-4628-81b3-6dce731a4f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:33.959837 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 919/1830] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:33.964527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e170bb8-5241-40a7-9822-c3b46fe6099c None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:33.973106 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-6e170bb8-5241-40a7-9822-c3b46fe6099c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:33.983347 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:34.004032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e170bb8-5241-40a7-9822-c3b46fe6099c None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:34.007559 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 912/1831] 10.222.0.27 () {58 vars in 1018 bytes} [Sat Jul 16 15:11:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:34.023474 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a0fab-d7ec-4713-973d-b122593cc9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:34.024560 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a0fab-d7ec-4713-973d-b122593cc9f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:34.025129 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a0fab-d7ec-4713-973d-b122593cc9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:34.025514 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a0fab-d7ec-4713-973d-b122593cc9f6 admin admin] PATH_INFO: `/v3/projects/9d645d5b2f5c4e2baa5b1a6d214a95b9/users/cf60bf4e17934b2fa3aa409073982dcd/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:34.028687 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a0fab-d7ec-4713-973d-b122593cc9f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d645d5b2f5c4e2baa5b1a6d214a95b9, user_id=cf60bf4e17934b2fa3aa409073982dcd, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:34.039208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a0fab-d7ec-4713-973d-b122593cc9f6 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:34.089785 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 920/1832] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:11:33 2022] PUT /identity/v3/projects/9d645d5b2f5c4e2baa5b1a6d214a95b9/users/cf60bf4e17934b2fa3aa409073982dcd/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:34.094094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadfdee-1f1e-4628-81b3-6dce731a4f25 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:34.106445 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-efadfdee-1f1e-4628-81b3-6dce731a4f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:34.114996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9725-fc3a-4ed5-a5a9-592bcf0d938f None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:34.115502 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9725-fc3a-4ed5-a5a9-592bcf0d938f None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:34.116598 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9725-fc3a-4ed5-a5a9-592bcf0d938f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:34.167548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-904f9725-fc3a-4ed5-a5a9-592bcf0d938f None None] MFA Rules not processed for user `cf60bf4e17934b2fa3aa409073982dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:34.169093 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-904f9725-fc3a-4ed5-a5a9-592bcf0d938f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:34.220180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadfdee-1f1e-4628-81b3-6dce731a4f25 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:34.321784 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 921/1833] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:34 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:34.321784 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f9725-fc3a-4ed5-a5a9-592bcf0d938f None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:34.321784 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-904f9725-fc3a-4ed5-a5a9-592bcf0d938f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:34.340745 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10d87c-f2a5-46f3-acc4-cb383cd4044b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:34.341312 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10d87c-f2a5-46f3-acc4-cb383cd4044b service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:34.341602 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10d87c-f2a5-46f3-acc4-cb383cd4044b service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:34.341882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10d87c-f2a5-46f3-acc4-cb383cd4044b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:34.460966 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10d87c-f2a5-46f3-acc4-cb383cd4044b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:34.462471 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10d87c-f2a5-46f3-acc4-cb383cd4044b service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:34.475803 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 913/1834] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:11:34.477152 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10d87c-f2a5-46f3-acc4-cb383cd4044b service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:34.480787 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5d10d87c-f2a5-46f3-acc4-cb383cd4044b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:34.492431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f9725-fc3a-4ed5-a5a9-592bcf0d938f None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:34.506513 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096fb22-4e70-42ba-936e-b44ba1a5c3cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:34.507900 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096fb22-4e70-42ba-936e-b44ba1a5c3cd service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:34.508248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096fb22-4e70-42ba-936e-b44ba1a5c3cd service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:34.508534 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096fb22-4e70-42ba-936e-b44ba1a5c3cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:34.520889 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096fb22-4e70-42ba-936e-b44ba1a5c3cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:34.523506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096fb22-4e70-42ba-936e-b44ba1a5c3cd service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:34.539596 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 922/1835] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:11:34.626256 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10d87c-f2a5-46f3-acc4-cb383cd4044b service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:34.633650 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a226a-41a7-4491-92bf-d973c487b61a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:34.636259 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a226a-41a7-4491-92bf-d973c487b61a service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:34.636763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a226a-41a7-4491-92bf-d973c487b61a service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:34.643310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a226a-41a7-4491-92bf-d973c487b61a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:34.653004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a226a-41a7-4491-92bf-d973c487b61a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:34.654816 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a226a-41a7-4491-92bf-d973c487b61a service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:34.669474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 914/1836] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:34 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:11:34.895907 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:35.122170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096fb22-4e70-42ba-936e-b44ba1a5c3cd service nova] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:35.146693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096fb22-4e70-42ba-936e-b44ba1a5c3cd service nova] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:35.156521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975cbd6-b2fc-4a41-a30d-ddae446c48b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:35.157897 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975cbd6-b2fc-4a41-a30d-ddae446c48b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:35.159145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975cbd6-b2fc-4a41-a30d-ddae446c48b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:35.159930 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975cbd6-b2fc-4a41-a30d-ddae446c48b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:35.163569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7975cbd6-b2fc-4a41-a30d-ddae446c48b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:35.301052 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:35.375408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7975cbd6-b2fc-4a41-a30d-ddae446c48b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:35.379701 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7975cbd6-b2fc-4a41-a30d-ddae446c48b9 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:35.399476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 923/1837] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:35 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:11:35.598048 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a226a-41a7-4491-92bf-d973c487b61a service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:35.599881 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6e4a226a-41a7-4491-92bf-d973c487b61a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:35.633617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a226a-41a7-4491-92bf-d973c487b61a service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:35.644778 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b39d3-bede-4ef0-8353-ddda674a2125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:35.646705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b39d3-bede-4ef0-8353-ddda674a2125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:35.647462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b39d3-bede-4ef0-8353-ddda674a2125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:35.648355 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b39d3-bede-4ef0-8353-ddda674a2125 admin admin] PATH_INFO: `/v3/projects/c2a9fd6b8a024d37919a3ed2138e8174` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:35.651343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b39d3-bede-4ef0-8353-ddda674a2125 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2a9fd6b8a024d37919a3ed2138e8174)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:35.655660 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b39d3-bede-4ef0-8353-ddda674a2125 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:35.934064 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ca5b39d3-bede-4ef0-8353-ddda674a2125 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2a9fd6b8a024d37919a3ed2138e8174', 'request_id': 'req-ca5b39d3-bede-4ef0-8353-ddda674a2125'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:35.942205 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ca5b39d3-bede-4ef0-8353-ddda674a2125 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2a9fd6b8a024d37919a3ed2138e8174', 'request_id': 'req-ca5b39d3-bede-4ef0-8353-ddda674a2125'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:35.949937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ca5b39d3-bede-4ef0-8353-ddda674a2125 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2a9fd6b8a024d37919a3ed2138e8174', 'request_id': 'req-ca5b39d3-bede-4ef0-8353-ddda674a2125'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:36.053254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ca5b39d3-bede-4ef0-8353-ddda674a2125 admin admin] The token cache is being invalidate because project c2a9fd6b8a024d37919a3ed2138e8174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:11:36.053954 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ca5b39d3-bede-4ef0-8353-ddda674a2125 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:36.057622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ca5b39d3-bede-4ef0-8353-ddda674a2125 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:36.072249 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 915/1838] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:11:35 2022] DELETE /identity/v3/projects/c2a9fd6b8a024d37919a3ed2138e8174 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:36.078135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975cbd6-b2fc-4a41-a30d-ddae446c48b9 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:36.099883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975cbd6-b2fc-4a41-a30d-ddae446c48b9 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:36.229165 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75df763-22c6-475f-ab38-65246febd7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:36.231131 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75df763-22c6-475f-ab38-65246febd7ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:36.231874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75df763-22c6-475f-ab38-65246febd7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:36.232379 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75df763-22c6-475f-ab38-65246febd7ab admin admin] PATH_INFO: `/v3/users/de2311da1bf74dcba57c946e3342fe34` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:36.235043 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75df763-22c6-475f-ab38-65246febd7ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=de2311da1bf74dcba57c946e3342fe34)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:36.240158 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75df763-22c6-475f-ab38-65246febd7ab admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:36.498400 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c75df763-22c6-475f-ab38-65246febd7ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de2311da1bf74dcba57c946e3342fe34'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:36.585830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c75df763-22c6-475f-ab38-65246febd7ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de2311da1bf74dcba57c946e3342fe34'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:36.605474 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c75df763-22c6-475f-ab38-65246febd7ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de2311da1bf74dcba57c946e3342fe34'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:36.625819 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c75df763-22c6-475f-ab38-65246febd7ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de2311da1bf74dcba57c946e3342fe34'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:36.626754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c75df763-22c6-475f-ab38-65246febd7ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de2311da1bf74dcba57c946e3342fe34'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:36.638480 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 924/1839] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:11:36 2022] DELETE /identity/v3/users/de2311da1bf74dcba57c946e3342fe34 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:11:37.387336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:37.388409 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b39d3-bede-4ef0-8353-ddda674a2125 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:37.389672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ca5b39d3-bede-4ef0-8353-ddda674a2125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:37.406082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b39d3-bede-4ef0-8353-ddda674a2125 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:37.483154 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03338b-b9bb-422f-addc-fd14cc25e816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:37.484456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03338b-b9bb-422f-addc-fd14cc25e816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:37.484842 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03338b-b9bb-422f-addc-fd14cc25e816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:37.485259 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03338b-b9bb-422f-addc-fd14cc25e816 admin admin] PATH_INFO: `/v3/projects/b68e8232db774f76b002decbf0f34361` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:37.486969 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a03338b-b9bb-422f-addc-fd14cc25e816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b68e8232db774f76b002decbf0f34361)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:37.489267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a03338b-b9bb-422f-addc-fd14cc25e816 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:37.660243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3a03338b-b9bb-422f-addc-fd14cc25e816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b68e8232db774f76b002decbf0f34361', 'request_id': 'req-3a03338b-b9bb-422f-addc-fd14cc25e816'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:37.671245 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3a03338b-b9bb-422f-addc-fd14cc25e816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b68e8232db774f76b002decbf0f34361', 'request_id': 'req-3a03338b-b9bb-422f-addc-fd14cc25e816'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:37.684127 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3a03338b-b9bb-422f-addc-fd14cc25e816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b68e8232db774f76b002decbf0f34361', 'request_id': 'req-3a03338b-b9bb-422f-addc-fd14cc25e816'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:37.691526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:37.798876 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3a03338b-b9bb-422f-addc-fd14cc25e816 admin admin] The token cache is being invalidate because project b68e8232db774f76b002decbf0f34361 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:11:37.799209 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3a03338b-b9bb-422f-addc-fd14cc25e816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:37.799981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3a03338b-b9bb-422f-addc-fd14cc25e816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:37.802545 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 916/1840] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:11:37 2022] DELETE /identity/v3/projects/b68e8232db774f76b002decbf0f34361 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:37.808273 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215f0f7-a64c-40a1-a2c5-ba8c21e75ba7 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:37.808627 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215f0f7-a64c-40a1-a2c5-ba8c21e75ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:37.809098 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215f0f7-a64c-40a1-a2c5-ba8c21e75ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:37.840654 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-1215f0f7-a64c-40a1-a2c5-ba8c21e75ba7 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:37.842847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-1215f0f7-a64c-40a1-a2c5-ba8c21e75ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:37.995266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 925/1841] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:11:38.009712 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92066d1a-1db8-451c-9c87-2d08529e25f8 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:38.010120 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92066d1a-1db8-451c-9c87-2d08529e25f8 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:38.010636 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92066d1a-1db8-451c-9c87-2d08529e25f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:38.046844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-92066d1a-1db8-451c-9c87-2d08529e25f8 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:38.049083 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-92066d1a-1db8-451c-9c87-2d08529e25f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:38.065653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 917/1842] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:38.071922 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215f0f7-a64c-40a1-a2c5-ba8c21e75ba7 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:38.094009 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215f0f7-a64c-40a1-a2c5-ba8c21e75ba7 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:38.100678 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96183e-47e1-4b2d-8cac-1c016fa408e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:38.101764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96183e-47e1-4b2d-8cac-1c016fa408e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:38.102251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96183e-47e1-4b2d-8cac-1c016fa408e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:38.102631 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b96183e-47e1-4b2d-8cac-1c016fa408e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:38.106456 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96183e-47e1-4b2d-8cac-1c016fa408e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:11:38.107022 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96183e-47e1-4b2d-8cac-1c016fa408e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:38.110977 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b96183e-47e1-4b2d-8cac-1c016fa408e2 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:38.149267 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 926/1843] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:11:38.156188 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92066d1a-1db8-451c-9c87-2d08529e25f8 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:38.177746 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92066d1a-1db8-451c-9c87-2d08529e25f8 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:38.182286 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bb49d-4470-4052-829f-9880c5eefc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:38.183479 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bb49d-4470-4052-829f-9880c5eefc13 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:38.183982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bb49d-4470-4052-829f-9880c5eefc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:38.185086 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5bb49d-4470-4052-829f-9880c5eefc13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:38.185830 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5bb49d-4470-4052-829f-9880c5eefc13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:38.190223 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5bb49d-4470-4052-829f-9880c5eefc13 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:38.349586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:38.356825 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 918/1844] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:11:38 2022] POST /identity/v3/projects => generated 355 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:11:38.360783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96183e-47e1-4b2d-8cac-1c016fa408e2 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:38.371990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b96183e-47e1-4b2d-8cac-1c016fa408e2 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:38.440578 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2775c5-cee7-491a-a400-2c62555af9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:38.442157 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2775c5-cee7-491a-a400-2c62555af9ad admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:38.442777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2775c5-cee7-491a-a400-2c62555af9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:38.443246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2775c5-cee7-491a-a400-2c62555af9ad admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:38.445506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2775c5-cee7-491a-a400-2c62555af9ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:38.448100 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2775c5-cee7-491a-a400-2c62555af9ad admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:38.590251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 927/1845] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:11:38 2022] POST /identity/v3/users => generated 278 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:11:38.595103 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bb49d-4470-4052-829f-9880c5eefc13 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:38.612190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5bb49d-4470-4052-829f-9880c5eefc13 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:38.618654 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8022f-0f36-4848-ace9-ba1679b6e1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:38.619871 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8022f-0f36-4848-ace9-ba1679b6e1ec admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:38.625578 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8022f-0f36-4848-ace9-ba1679b6e1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:38.625578 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8022f-0f36-4848-ace9-ba1679b6e1ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:38.625578 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8022f-0f36-4848-ace9-ba1679b6e1ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:38.625578 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8022f-0f36-4848-ace9-ba1679b6e1ec admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:38.638944 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 919/1846] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:38.642866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2775c5-cee7-491a-a400-2c62555af9ad admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:38.653549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2775c5-cee7-491a-a400-2c62555af9ad admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:38.658505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38462ce-4be2-4659-9f24-5368309c4b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:38.659241 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38462ce-4be2-4659-9f24-5368309c4b0f admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:38.659787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38462ce-4be2-4659-9f24-5368309c4b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:38.660231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38462ce-4be2-4659-9f24-5368309c4b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:38.661142 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38462ce-4be2-4659-9f24-5368309c4b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:38.662806 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38462ce-4be2-4659-9f24-5368309c4b0f admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:38.674764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 928/1847] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:38.679634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8022f-0f36-4848-ace9-ba1679b6e1ec admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:38.692572 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8022f-0f36-4848-ace9-ba1679b6e1ec admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:38.698585 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463378a-b6f5-469d-8018-226e21bd5965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:38.699736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463378a-b6f5-469d-8018-226e21bd5965 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:38.706723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463378a-b6f5-469d-8018-226e21bd5965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:38.706723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463378a-b6f5-469d-8018-226e21bd5965 admin admin] PATH_INFO: `/v3/projects/23652ac7beb34b3a81a710205fe51102/users/9ebbef1f078044bfb8ebb741d2667da7/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:38.717860 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463378a-b6f5-469d-8018-226e21bd5965 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23652ac7beb34b3a81a710205fe51102, user_id=9ebbef1f078044bfb8ebb741d2667da7, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:38.722721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463378a-b6f5-469d-8018-226e21bd5965 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:38.843287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 920/1848] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:11:38 2022] PUT /identity/v3/projects/23652ac7beb34b3a81a710205fe51102/users/9ebbef1f078044bfb8ebb741d2667da7/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:38.854790 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5cbd3-e6a8-4fe7-9445-2956fd535bc6 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:38.855144 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5cbd3-e6a8-4fe7-9445-2956fd535bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:38.855640 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5cbd3-e6a8-4fe7-9445-2956fd535bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:38.904830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-efd5cbd3-e6a8-4fe7-9445-2956fd535bc6 None None] MFA Rules not processed for user `9ebbef1f078044bfb8ebb741d2667da7`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:38.906990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-efd5cbd3-e6a8-4fe7-9445-2956fd535bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:39.005744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 929/1849] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:38 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:11:39.011603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463378a-b6f5-469d-8018-226e21bd5965 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:39.134158 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463378a-b6f5-469d-8018-226e21bd5965 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:39.143954 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fe33a-d645-47e2-9b44-db4437f7f6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:39.145303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fe33a-d645-47e2-9b44-db4437f7f6c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:39.145903 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fe33a-d645-47e2-9b44-db4437f7f6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:39.146418 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fe33a-d645-47e2-9b44-db4437f7f6c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:39.147827 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fe33a-d645-47e2-9b44-db4437f7f6c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:39.151903 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fe33a-d645-47e2-9b44-db4437f7f6c7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:39.290339 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 921/1850] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:11:39 2022] POST /identity/v3/projects => generated 355 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:11:39.296454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5cbd3-e6a8-4fe7-9445-2956fd535bc6 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:39.309373 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5cbd3-e6a8-4fe7-9445-2956fd535bc6 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:39.345535 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:39.446315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434266a3-fc8f-45c3-ac0e-eefad25ee2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:39.448847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434266a3-fc8f-45c3-ac0e-eefad25ee2f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:39.449598 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434266a3-fc8f-45c3-ac0e-eefad25ee2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:39.450332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434266a3-fc8f-45c3-ac0e-eefad25ee2f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:39.452266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434266a3-fc8f-45c3-ac0e-eefad25ee2f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:39.457959 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434266a3-fc8f-45c3-ac0e-eefad25ee2f7 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:39.578972 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 930/1851] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:11:39 2022] POST /identity/v3/users => generated 284 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:11:39.584919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fe33a-d645-47e2-9b44-db4437f7f6c7 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:39.599221 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fe33a-d645-47e2-9b44-db4437f7f6c7 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:39.606107 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd600d-875e-42bb-ad43-c31197d6ec14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:39.608578 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd600d-875e-42bb-ad43-c31197d6ec14 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:39.608578 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd600d-875e-42bb-ad43-c31197d6ec14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:39.608578 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd600d-875e-42bb-ad43-c31197d6ec14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:39.608578 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd600d-875e-42bb-ad43-c31197d6ec14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:39.610814 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd600d-875e-42bb-ad43-c31197d6ec14 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:39.625606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 922/1852] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:39.631364 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434266a3-fc8f-45c3-ac0e-eefad25ee2f7 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:39.631675 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd600d-875e-42bb-ad43-c31197d6ec14 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:39.636295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5cdd600d-875e-42bb-ad43-c31197d6ec14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:39.648837 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434266a3-fc8f-45c3-ac0e-eefad25ee2f7 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:39.653286 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd600d-875e-42bb-ad43-c31197d6ec14 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:39.660039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75778044-c748-402e-988f-056a4555941d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:39.662024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75778044-c748-402e-988f-056a4555941d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:39.662786 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75778044-c748-402e-988f-056a4555941d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:39.663297 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75778044-c748-402e-988f-056a4555941d admin admin] PATH_INFO: `/v3/users/e111ab406c5140cd9d1bba56832c1ab2` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:39.666654 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75778044-c748-402e-988f-056a4555941d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e111ab406c5140cd9d1bba56832c1ab2)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:39.667653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8faf30-a7b6-41f4-b1cf-14d8f2326dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:39.668601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8faf30-a7b6-41f4-b1cf-14d8f2326dc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:39.668884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8faf30-a7b6-41f4-b1cf-14d8f2326dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:39.669160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8faf30-a7b6-41f4-b1cf-14d8f2326dc5 admin admin] PATH_INFO: `/v3/projects/c63a85df05644010a80d7c5876604b42/users/98576fede6284eb796a4b30d400eea9f/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:39.670555 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75778044-c748-402e-988f-056a4555941d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:39.719218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8faf30-a7b6-41f4-b1cf-14d8f2326dc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c63a85df05644010a80d7c5876604b42, user_id=98576fede6284eb796a4b30d400eea9f, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:39.722730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8faf30-a7b6-41f4-b1cf-14d8f2326dc5 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:39.915759 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 931/1853] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:11:39 2022] PUT /identity/v3/projects/c63a85df05644010a80d7c5876604b42/users/98576fede6284eb796a4b30d400eea9f/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:39.938307 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-75778044-c748-402e-988f-056a4555941d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e111ab406c5140cd9d1bba56832c1ab2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:39.943756 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8faf30-a7b6-41f4-b1cf-14d8f2326dc5 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:40.039787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8faf30-a7b6-41f4-b1cf-14d8f2326dc5 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:40.045076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624620a2-3669-4b4e-84c3-9a429048023a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:40.045854 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624620a2-3669-4b4e-84c3-9a429048023a admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:40.046146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624620a2-3669-4b4e-84c3-9a429048023a admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:40.046432 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624620a2-3669-4b4e-84c3-9a429048023a admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:40.047222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624620a2-3669-4b4e-84c3-9a429048023a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:40.049116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624620a2-3669-4b4e-84c3-9a429048023a admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:40.058737 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-75778044-c748-402e-988f-056a4555941d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e111ab406c5140cd9d1bba56832c1ab2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:40.059461 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-75778044-c748-402e-988f-056a4555941d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e111ab406c5140cd9d1bba56832c1ab2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:40.060156 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-75778044-c748-402e-988f-056a4555941d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e111ab406c5140cd9d1bba56832c1ab2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:40.067075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 932/1854] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:40.068196 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624620a2-3669-4b4e-84c3-9a429048023a admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:40.070473 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-624620a2-3669-4b4e-84c3-9a429048023a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:40.071693 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:40.087094 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624620a2-3669-4b4e-84c3-9a429048023a admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:40.088418 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-75778044-c748-402e-988f-056a4555941d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e111ab406c5140cd9d1bba56832c1ab2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:40.096143 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374d820-13cd-4511-8e86-7baac3a251fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:40.097493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374d820-13cd-4511-8e86-7baac3a251fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:40.097751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374d820-13cd-4511-8e86-7baac3a251fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:40.098765 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374d820-13cd-4511-8e86-7baac3a251fc admin admin] PATH_INFO: `/v3/projects/c63a85df05644010a80d7c5876604b42/users/98576fede6284eb796a4b30d400eea9f/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:40.101558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374d820-13cd-4511-8e86-7baac3a251fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c63a85df05644010a80d7c5876604b42, user_id=98576fede6284eb796a4b30d400eea9f, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:40.104970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374d820-13cd-4511-8e86-7baac3a251fc admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:40.120419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 933/1855] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:11:40 2022] PUT /identity/v3/projects/c63a85df05644010a80d7c5876604b42/users/98576fede6284eb796a4b30d400eea9f/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:40.122421 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 923/1856] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:11:39 2022] DELETE /identity/v3/users/e111ab406c5140cd9d1bba56832c1ab2 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:40.129483 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01232b-03b0-4ed6-b122-c00588e9e873 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:40.129942 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01232b-03b0-4ed6-b122-c00588e9e873 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:40.130835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01232b-03b0-4ed6-b122-c00588e9e873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:40.133927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08174997-ce6d-43cd-9b90-c3996fb4b777 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:40.134417 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08174997-ce6d-43cd-9b90-c3996fb4b777 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:40.134963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08174997-ce6d-43cd-9b90-c3996fb4b777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:40.155802 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-5e01232b-03b0-4ed6-b122-c00588e9e873 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:40.157274 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5e01232b-03b0-4ed6-b122-c00588e9e873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:40.181192 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-08174997-ce6d-43cd-9b90-c3996fb4b777 None None] MFA Rules not processed for user `98576fede6284eb796a4b30d400eea9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:40.182457 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-08174997-ce6d-43cd-9b90-c3996fb4b777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:40.232711 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 934/1857] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:11:40.240305 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01232b-03b0-4ed6-b122-c00588e9e873 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:40.241817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5e01232b-03b0-4ed6-b122-c00588e9e873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:40.264763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 924/1858] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:40 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:40.271122 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:40.294120 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08174997-ce6d-43cd-9b90-c3996fb4b777 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:40.296630 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-08174997-ce6d-43cd-9b90-c3996fb4b777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:40.327511 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01232b-03b0-4ed6-b122-c00588e9e873 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:40.332897 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acab696-7888-4ec2-bd35-7428bd2a8fcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:40.333702 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acab696-7888-4ec2-bd35-7428bd2a8fcd service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:40.334051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acab696-7888-4ec2-bd35-7428bd2a8fcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:40.334398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acab696-7888-4ec2-bd35-7428bd2a8fcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:40.347761 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acab696-7888-4ec2-bd35-7428bd2a8fcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:40.351702 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acab696-7888-4ec2-bd35-7428bd2a8fcd service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:40.376425 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 935/1859] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:11:40.409545 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08174997-ce6d-43cd-9b90-c3996fb4b777 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:40.419960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea9fd7-8a39-44e4-aa22-af5f7da36930 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:40.422808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea9fd7-8a39-44e4-aa22-af5f7da36930 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:40.428663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea9fd7-8a39-44e4-aa22-af5f7da36930 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:40.428663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea9fd7-8a39-44e4-aa22-af5f7da36930 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:40.449673 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea9fd7-8a39-44e4-aa22-af5f7da36930 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:40.455716 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea9fd7-8a39-44e4-aa22-af5f7da36930 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:40.468402 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 925/1860] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:40 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:11:40.799946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acab696-7888-4ec2-bd35-7428bd2a8fcd service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:40.808040 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-2acab696-7888-4ec2-bd35-7428bd2a8fcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:40.832287 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acab696-7888-4ec2-bd35-7428bd2a8fcd service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:40.843341 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8437ab-f264-4ea4-a721-e5380dc52acb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:40.844794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8437ab-f264-4ea4-a721-e5380dc52acb service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:40.846203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8437ab-f264-4ea4-a721-e5380dc52acb service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:40.846203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8437ab-f264-4ea4-a721-e5380dc52acb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:40.854500 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-2c8437ab-f264-4ea4-a721-e5380dc52acb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:41.019026 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8437ab-f264-4ea4-a721-e5380dc52acb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:41.021769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8437ab-f264-4ea4-a721-e5380dc52acb service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:41.041814 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 936/1861] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:40 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:11:41.403688 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea9fd7-8a39-44e4-aa22-af5f7da36930 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:41.405586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-58ea9fd7-8a39-44e4-aa22-af5f7da36930 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:41.422090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea9fd7-8a39-44e4-aa22-af5f7da36930 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:41.429413 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889cd3a-79d8-4c24-b229-70d58e923c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:41.430213 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889cd3a-79d8-4c24-b229-70d58e923c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:41.430685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889cd3a-79d8-4c24-b229-70d58e923c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:41.436350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889cd3a-79d8-4c24-b229-70d58e923c5b admin admin] PATH_INFO: `/v3/projects/a18f9103b143427c8c2d52cfc3d19688` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:41.438101 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889cd3a-79d8-4c24-b229-70d58e923c5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a18f9103b143427c8c2d52cfc3d19688)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:41.440464 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889cd3a-79d8-4c24-b229-70d58e923c5b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:41.612121 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3889cd3a-79d8-4c24-b229-70d58e923c5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a18f9103b143427c8c2d52cfc3d19688', 'request_id': 'req-3889cd3a-79d8-4c24-b229-70d58e923c5b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:41.618390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3889cd3a-79d8-4c24-b229-70d58e923c5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a18f9103b143427c8c2d52cfc3d19688', 'request_id': 'req-3889cd3a-79d8-4c24-b229-70d58e923c5b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:41.620760 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:41.626222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3889cd3a-79d8-4c24-b229-70d58e923c5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a18f9103b143427c8c2d52cfc3d19688', 'request_id': 'req-3889cd3a-79d8-4c24-b229-70d58e923c5b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:41.730965 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3889cd3a-79d8-4c24-b229-70d58e923c5b admin admin] The token cache is being invalidate because project a18f9103b143427c8c2d52cfc3d19688 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:11:41.730965 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3889cd3a-79d8-4c24-b229-70d58e923c5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:41.730965 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3889cd3a-79d8-4c24-b229-70d58e923c5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:41.734052 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 926/1862] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:11:41 2022] DELETE /identity/v3/projects/a18f9103b143427c8c2d52cfc3d19688 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:41.741957 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8437ab-f264-4ea4-a721-e5380dc52acb service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:41.759766 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8437ab-f264-4ea4-a721-e5380dc52acb service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:41.841372 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d9c64-26c9-410f-83df-1dcbe78b8011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:41.842360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d9c64-26c9-410f-83df-1dcbe78b8011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:41.842852 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d9c64-26c9-410f-83df-1dcbe78b8011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:41.843416 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d9c64-26c9-410f-83df-1dcbe78b8011 admin admin] PATH_INFO: `/v3/users/cf60bf4e17934b2fa3aa409073982dcd` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:41.844941 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d9c64-26c9-410f-83df-1dcbe78b8011 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf60bf4e17934b2fa3aa409073982dcd)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:41.848194 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d9c64-26c9-410f-83df-1dcbe78b8011 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:42.107394 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5e4d9c64-26c9-410f-83df-1dcbe78b8011 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf60bf4e17934b2fa3aa409073982dcd'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:42.194508 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5e4d9c64-26c9-410f-83df-1dcbe78b8011 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf60bf4e17934b2fa3aa409073982dcd'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:42.199838 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:42.211251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5e4d9c64-26c9-410f-83df-1dcbe78b8011 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf60bf4e17934b2fa3aa409073982dcd'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:42.231425 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5e4d9c64-26c9-410f-83df-1dcbe78b8011 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf60bf4e17934b2fa3aa409073982dcd'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:42.232588 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5e4d9c64-26c9-410f-83df-1dcbe78b8011 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf60bf4e17934b2fa3aa409073982dcd'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:42.245406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 937/1863] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:11:41 2022] DELETE /identity/v3/users/cf60bf4e17934b2fa3aa409073982dcd => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:11:43.580441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889cd3a-79d8-4c24-b229-70d58e923c5b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:43.580441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3889cd3a-79d8-4c24-b229-70d58e923c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:43.607697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889cd3a-79d8-4c24-b229-70d58e923c5b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:43.688574 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52558bf7-06b9-4c74-8f04-3db69a43c31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:43.689517 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52558bf7-06b9-4c74-8f04-3db69a43c31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:43.689849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52558bf7-06b9-4c74-8f04-3db69a43c31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:43.690193 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52558bf7-06b9-4c74-8f04-3db69a43c31f admin admin] PATH_INFO: `/v3/projects/9d645d5b2f5c4e2baa5b1a6d214a95b9` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:43.691666 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52558bf7-06b9-4c74-8f04-3db69a43c31f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d645d5b2f5c4e2baa5b1a6d214a95b9)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:43.694527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52558bf7-06b9-4c74-8f04-3db69a43c31f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:43.873942 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-52558bf7-06b9-4c74-8f04-3db69a43c31f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d645d5b2f5c4e2baa5b1a6d214a95b9', 'request_id': 'req-52558bf7-06b9-4c74-8f04-3db69a43c31f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:43.886616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-52558bf7-06b9-4c74-8f04-3db69a43c31f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d645d5b2f5c4e2baa5b1a6d214a95b9', 'request_id': 'req-52558bf7-06b9-4c74-8f04-3db69a43c31f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:43.896511 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-52558bf7-06b9-4c74-8f04-3db69a43c31f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d645d5b2f5c4e2baa5b1a6d214a95b9', 'request_id': 'req-52558bf7-06b9-4c74-8f04-3db69a43c31f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:44.120111 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-52558bf7-06b9-4c74-8f04-3db69a43c31f admin admin] The token cache is being invalidate because project 9d645d5b2f5c4e2baa5b1a6d214a95b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:11:44.120946 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-52558bf7-06b9-4c74-8f04-3db69a43c31f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:44.122077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-52558bf7-06b9-4c74-8f04-3db69a43c31f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:44.124231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 927/1864] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:11:43 2022] DELETE /identity/v3/projects/9d645d5b2f5c4e2baa5b1a6d214a95b9 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:44.131758 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23623f1c-9ab5-44fa-9699-483f44b538ff None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:44.132558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23623f1c-9ab5-44fa-9699-483f44b538ff None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:44.132558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23623f1c-9ab5-44fa-9699-483f44b538ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:44.170051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-23623f1c-9ab5-44fa-9699-483f44b538ff None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:44.171840 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-23623f1c-9ab5-44fa-9699-483f44b538ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:44.265041 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 938/1865] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:44.275219 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7624a50-c5cf-4674-a777-0ac6b64f8113 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:44.275806 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7624a50-c5cf-4674-a777-0ac6b64f8113 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:44.275997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7624a50-c5cf-4674-a777-0ac6b64f8113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:44.308274 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a7624a50-c5cf-4674-a777-0ac6b64f8113 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:44.310315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a7624a50-c5cf-4674-a777-0ac6b64f8113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:44.327940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 928/1866] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:44.329429 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23623f1c-9ab5-44fa-9699-483f44b538ff None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:44.345004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23623f1c-9ab5-44fa-9699-483f44b538ff None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:44.353643 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb777cc1-8534-4d7b-8bc2-c36088f47fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:44.354846 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb777cc1-8534-4d7b-8bc2-c36088f47fdb admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:44.355338 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb777cc1-8534-4d7b-8bc2-c36088f47fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:44.355808 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb777cc1-8534-4d7b-8bc2-c36088f47fdb admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:44.357101 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb777cc1-8534-4d7b-8bc2-c36088f47fdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:11:44.358683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb777cc1-8534-4d7b-8bc2-c36088f47fdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:44.360478 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb777cc1-8534-4d7b-8bc2-c36088f47fdb admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:44.381204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 939/1867] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:11:44.387896 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7624a50-c5cf-4674-a777-0ac6b64f8113 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:44.400890 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7624a50-c5cf-4674-a777-0ac6b64f8113 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:44.406248 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c207f54-128d-464e-959d-41993ce03f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:44.407043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c207f54-128d-464e-959d-41993ce03f6e admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:44.407341 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c207f54-128d-464e-959d-41993ce03f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:44.407637 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c207f54-128d-464e-959d-41993ce03f6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:44.409027 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c207f54-128d-464e-959d-41993ce03f6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:44.411157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c207f54-128d-464e-959d-41993ce03f6e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:44.533276 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 929/1868] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:11:44 2022] POST /identity/v3/projects => generated 375 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:11:44.539966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb777cc1-8534-4d7b-8bc2-c36088f47fdb admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:44.559342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb777cc1-8534-4d7b-8bc2-c36088f47fdb admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:44.651854 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff461ef5-be26-4990-86ce-812dd293fd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:44.653209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff461ef5-be26-4990-86ce-812dd293fd08 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:44.653702 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff461ef5-be26-4990-86ce-812dd293fd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:44.654206 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff461ef5-be26-4990-86ce-812dd293fd08 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:44.655616 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff461ef5-be26-4990-86ce-812dd293fd08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:44.659534 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff461ef5-be26-4990-86ce-812dd293fd08 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:44.987016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 940/1869] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:11:44 2022] POST /identity/v3/users => generated 288 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:11:44.991301 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c207f54-128d-464e-959d-41993ce03f6e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:45.007762 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c207f54-128d-464e-959d-41993ce03f6e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:45.014509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb02e1a-6e08-423e-88a6-7cd67c02b06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:45.015668 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb02e1a-6e08-423e-88a6-7cd67c02b06d admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:45.016153 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb02e1a-6e08-423e-88a6-7cd67c02b06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:45.016700 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb02e1a-6e08-423e-88a6-7cd67c02b06d admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:45.018282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb02e1a-6e08-423e-88a6-7cd67c02b06d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:45.021464 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb02e1a-6e08-423e-88a6-7cd67c02b06d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:45.045124 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 930/1870] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:44 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:45.049992 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff461ef5-be26-4990-86ce-812dd293fd08 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:45.064866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff461ef5-be26-4990-86ce-812dd293fd08 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:45.071689 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b3907-d3e6-4801-bf08-6358b16bee3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:45.072712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b3907-d3e6-4801-bf08-6358b16bee3b admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:45.073245 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b3907-d3e6-4801-bf08-6358b16bee3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:45.073781 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b3907-d3e6-4801-bf08-6358b16bee3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:45.074766 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b3907-d3e6-4801-bf08-6358b16bee3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:45.076741 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b3907-d3e6-4801-bf08-6358b16bee3b admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:45.118049 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 941/1871] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:11:45.123800 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb02e1a-6e08-423e-88a6-7cd67c02b06d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:45.142000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb02e1a-6e08-423e-88a6-7cd67c02b06d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:45.166730 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbca3bc-873f-44d0-a381-6068395f5bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:45.168246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbca3bc-873f-44d0-a381-6068395f5bc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:45.168683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbca3bc-873f-44d0-a381-6068395f5bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:45.169137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbca3bc-873f-44d0-a381-6068395f5bc4 admin admin] PATH_INFO: `/v3/projects/0e0a9eeca0b1458dbfaffb22c417f131/users/38398c84846b4d52a43365a1688db174/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:45.213657 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbca3bc-873f-44d0-a381-6068395f5bc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e0a9eeca0b1458dbfaffb22c417f131, user_id=38398c84846b4d52a43365a1688db174, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:45.218932 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbca3bc-873f-44d0-a381-6068395f5bc4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:45.305773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 931/1872] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:11:45 2022] PUT /identity/v3/projects/0e0a9eeca0b1458dbfaffb22c417f131/users/38398c84846b4d52a43365a1688db174/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:11:45.318072 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d517ed-e8ef-46c2-8bf9-0d585716d725 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:45.319531 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d517ed-e8ef-46c2-8bf9-0d585716d725 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:45.320041 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d517ed-e8ef-46c2-8bf9-0d585716d725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:45.358708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-51d517ed-e8ef-46c2-8bf9-0d585716d725 None None] MFA Rules not processed for user `38398c84846b4d52a43365a1688db174`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:45.360677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-51d517ed-e8ef-46c2-8bf9-0d585716d725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:45.399796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:45.450403 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 942/1873] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:46.207003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbca3bc-873f-44d0-a381-6068395f5bc4 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:46.211383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2fbca3bc-873f-44d0-a381-6068395f5bc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:46.278827 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:46.345090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbca3bc-873f-44d0-a381-6068395f5bc4 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:46.350768 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c27941-3485-4c65-a44c-84e1670f68c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:46.351982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c27941-3485-4c65-a44c-84e1670f68c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:46.352500 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c27941-3485-4c65-a44c-84e1670f68c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:46.352982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c27941-3485-4c65-a44c-84e1670f68c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:46.366268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c27941-3485-4c65-a44c-84e1670f68c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:46.368209 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c27941-3485-4c65-a44c-84e1670f68c7 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:46.381194 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 932/1874] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:11:48.355249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d517ed-e8ef-46c2-8bf9-0d585716d725 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:48.357932 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-51d517ed-e8ef-46c2-8bf9-0d585716d725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:48.547130 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d517ed-e8ef-46c2-8bf9-0d585716d725 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:48.561889 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca04f0-4e62-4690-ad31-83e04eb246e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:48.568965 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca04f0-4e62-4690-ad31-83e04eb246e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:48.569554 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca04f0-4e62-4690-ad31-83e04eb246e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:48.570416 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca04f0-4e62-4690-ad31-83e04eb246e3 admin admin] PATH_INFO: `/v3/users/d0ff150bf5e8404c90e1e9fbe85fffe5` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:48.573526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cca04f0-4e62-4690-ad31-83e04eb246e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0ff150bf5e8404c90e1e9fbe85fffe5)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:48.577729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cca04f0-4e62-4690-ad31-83e04eb246e3 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:48.870664 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-1cca04f0-4e62-4690-ad31-83e04eb246e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0ff150bf5e8404c90e1e9fbe85fffe5'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:48.969016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-1cca04f0-4e62-4690-ad31-83e04eb246e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0ff150bf5e8404c90e1e9fbe85fffe5'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:48.996425 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-1cca04f0-4e62-4690-ad31-83e04eb246e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0ff150bf5e8404c90e1e9fbe85fffe5'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:49.034710 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-1cca04f0-4e62-4690-ad31-83e04eb246e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0ff150bf5e8404c90e1e9fbe85fffe5'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:49.036403 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-1cca04f0-4e62-4690-ad31-83e04eb246e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0ff150bf5e8404c90e1e9fbe85fffe5'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:49.051232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 943/1875] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:11:48 2022] DELETE /identity/v3/users/d0ff150bf5e8404c90e1e9fbe85fffe5 => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:49.057954 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736e8c5-5e02-4966-be22-e38bdac7051c None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:49.059524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736e8c5-5e02-4966-be22-e38bdac7051c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:49.060022 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736e8c5-5e02-4966-be22-e38bdac7051c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:49.104345 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-7736e8c5-5e02-4966-be22-e38bdac7051c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:49.106459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7736e8c5-5e02-4966-be22-e38bdac7051c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:49.277567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 933/1876] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:11:49.307238 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca04f0-4e62-4690-ad31-83e04eb246e3 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:49.309937 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-1cca04f0-4e62-4690-ad31-83e04eb246e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:49.394714 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:49.399121 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca04f0-4e62-4690-ad31-83e04eb246e3 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:49.403325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7276c7-d0a2-4e78-9e41-1e39b38b2b14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:49.404055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7276c7-d0a2-4e78-9e41-1e39b38b2b14 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:49.404340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7276c7-d0a2-4e78-9e41-1e39b38b2b14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:49.404621 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7276c7-d0a2-4e78-9e41-1e39b38b2b14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:49.434382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7276c7-d0a2-4e78-9e41-1e39b38b2b14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:49.435919 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7276c7-d0a2-4e78-9e41-1e39b38b2b14 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:49.479476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 944/1877] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:11:50.812918 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736e8c5-5e02-4966-be22-e38bdac7051c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:50.816776 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7736e8c5-5e02-4966-be22-e38bdac7051c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:50.865200 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736e8c5-5e02-4966-be22-e38bdac7051c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:50.888627 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4214e-6fce-47b2-9905-8908e5156b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:50.890330 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4214e-6fce-47b2-9905-8908e5156b5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:50.890837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4214e-6fce-47b2-9905-8908e5156b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:50.891376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4214e-6fce-47b2-9905-8908e5156b5b admin admin] PATH_INFO: `/v3/projects/16830c8c8eea4169bb6d7aeae2d051e1` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:50.894078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4214e-6fce-47b2-9905-8908e5156b5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=16830c8c8eea4169bb6d7aeae2d051e1)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:50.900293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4214e-6fce-47b2-9905-8908e5156b5b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:51.126965 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-26f4214e-6fce-47b2-9905-8908e5156b5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16830c8c8eea4169bb6d7aeae2d051e1', 'request_id': 'req-26f4214e-6fce-47b2-9905-8908e5156b5b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:51.135690 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-26f4214e-6fce-47b2-9905-8908e5156b5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16830c8c8eea4169bb6d7aeae2d051e1', 'request_id': 'req-26f4214e-6fce-47b2-9905-8908e5156b5b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:51.144515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-26f4214e-6fce-47b2-9905-8908e5156b5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16830c8c8eea4169bb6d7aeae2d051e1', 'request_id': 'req-26f4214e-6fce-47b2-9905-8908e5156b5b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:51.262361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-26f4214e-6fce-47b2-9905-8908e5156b5b admin admin] The token cache is being invalidate because project 16830c8c8eea4169bb6d7aeae2d051e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:11:51.262731 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-26f4214e-6fce-47b2-9905-8908e5156b5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:51.263507 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-26f4214e-6fce-47b2-9905-8908e5156b5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:51.266061 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 934/1878] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:11:50 2022] DELETE /identity/v3/projects/16830c8c8eea4169bb6d7aeae2d051e1 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:51.271473 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7276c7-d0a2-4e78-9e41-1e39b38b2b14 service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:51.286820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7276c7-d0a2-4e78-9e41-1e39b38b2b14 service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:51.365156 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a10b7-df68-4506-9723-4228110b1df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:51.366066 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a10b7-df68-4506-9723-4228110b1df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:51.366350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a10b7-df68-4506-9723-4228110b1df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:51.366627 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a10b7-df68-4506-9723-4228110b1df0 admin admin] PATH_INFO: `/v3/users/64b055cfc9be4b4bb892f09eaba782ca` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:51.367989 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a10b7-df68-4506-9723-4228110b1df0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64b055cfc9be4b4bb892f09eaba782ca)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:51.369996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a10b7-df68-4506-9723-4228110b1df0 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:51.601474 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-595a10b7-df68-4506-9723-4228110b1df0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64b055cfc9be4b4bb892f09eaba782ca'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:51.727856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-595a10b7-df68-4506-9723-4228110b1df0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64b055cfc9be4b4bb892f09eaba782ca'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:51.739068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-595a10b7-df68-4506-9723-4228110b1df0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64b055cfc9be4b4bb892f09eaba782ca'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:51.761002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-595a10b7-df68-4506-9723-4228110b1df0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64b055cfc9be4b4bb892f09eaba782ca'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:51.761786 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-595a10b7-df68-4506-9723-4228110b1df0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64b055cfc9be4b4bb892f09eaba782ca'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:51.770054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 945/1879] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:11:51 2022] DELETE /identity/v3/users/64b055cfc9be4b4bb892f09eaba782ca => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:11:52.322056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4214e-6fce-47b2-9905-8908e5156b5b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:52.323737 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-26f4214e-6fce-47b2-9905-8908e5156b5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:52.348435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4214e-6fce-47b2-9905-8908e5156b5b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:52.415187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:52.438391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f53732-35df-4de4-bf76-4c44ca0b044c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:52.440343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f53732-35df-4de4-bf76-4c44ca0b044c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:52.440824 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f53732-35df-4de4-bf76-4c44ca0b044c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:52.441314 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f53732-35df-4de4-bf76-4c44ca0b044c admin admin] PATH_INFO: `/v3/projects/315f9034b39d43c98ef9064b7da5ff3e` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:52.443906 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f53732-35df-4de4-bf76-4c44ca0b044c admin admin] RBAC: Authorizing `identity:delete_project(project_id=315f9034b39d43c98ef9064b7da5ff3e)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:52.447800 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f53732-35df-4de4-bf76-4c44ca0b044c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:52.571568 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-42f53732-35df-4de4-bf76-4c44ca0b044c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '315f9034b39d43c98ef9064b7da5ff3e', 'request_id': 'req-42f53732-35df-4de4-bf76-4c44ca0b044c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:52.578301 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-42f53732-35df-4de4-bf76-4c44ca0b044c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '315f9034b39d43c98ef9064b7da5ff3e', 'request_id': 'req-42f53732-35df-4de4-bf76-4c44ca0b044c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:52.586488 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-42f53732-35df-4de4-bf76-4c44ca0b044c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '315f9034b39d43c98ef9064b7da5ff3e', 'request_id': 'req-42f53732-35df-4de4-bf76-4c44ca0b044c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:52.663427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-42f53732-35df-4de4-bf76-4c44ca0b044c admin admin] The token cache is being invalidate because project 315f9034b39d43c98ef9064b7da5ff3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:11:52.663726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-42f53732-35df-4de4-bf76-4c44ca0b044c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:52.664681 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-42f53732-35df-4de4-bf76-4c44ca0b044c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:52.667535 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 935/1880] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:11:52 2022] DELETE /identity/v3/projects/315f9034b39d43c98ef9064b7da5ff3e => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:11:52.678953 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4d57f-1a53-4d0f-87a2-63961840f2fa None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:52.680275 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4d57f-1a53-4d0f-87a2-63961840f2fa None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:52.681020 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4d57f-1a53-4d0f-87a2-63961840f2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:52.707515 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-dac4d57f-1a53-4d0f-87a2-63961840f2fa None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:52.708907 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-dac4d57f-1a53-4d0f-87a2-63961840f2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:52.793222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 946/1881] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:52.802707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b931943-109c-4982-9520-c2f9699df336 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:52.803024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b931943-109c-4982-9520-c2f9699df336 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:52.803312 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b931943-109c-4982-9520-c2f9699df336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:52.823173 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5b931943-109c-4982-9520-c2f9699df336 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:52.824417 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5b931943-109c-4982-9520-c2f9699df336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:52.833557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 936/1882] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:52.838470 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4d57f-1a53-4d0f-87a2-63961840f2fa None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:52.853743 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4d57f-1a53-4d0f-87a2-63961840f2fa None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:52.862284 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77eed1-9fdb-464b-ad43-ce524b904419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:52.863462 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77eed1-9fdb-464b-ad43-ce524b904419 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:52.863957 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77eed1-9fdb-464b-ad43-ce524b904419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:52.864450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77eed1-9fdb-464b-ad43-ce524b904419 admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:52.865773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77eed1-9fdb-464b-ad43-ce524b904419 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:11:52.866320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77eed1-9fdb-464b-ad43-ce524b904419 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:52.869531 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77eed1-9fdb-464b-ad43-ce524b904419 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:52.883675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:52.894210 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 947/1883] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:11:52.898715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b931943-109c-4982-9520-c2f9699df336 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:52.909127 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b931943-109c-4982-9520-c2f9699df336 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:52.914239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902dd81-9691-463b-9a01-2e67320f7f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:52.915326 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902dd81-9691-463b-9a01-2e67320f7f2b admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:52.915663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902dd81-9691-463b-9a01-2e67320f7f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:52.915980 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902dd81-9691-463b-9a01-2e67320f7f2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:52.916726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902dd81-9691-463b-9a01-2e67320f7f2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:52.918761 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902dd81-9691-463b-9a01-2e67320f7f2b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:53.021265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 937/1884] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:11:52 2022] POST /identity/v3/projects => generated 363 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:11:53.025275 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77eed1-9fdb-464b-ad43-ce524b904419 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:53.036560 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77eed1-9fdb-464b-ad43-ce524b904419 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:53.118537 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0128a33-89e8-4c46-aa68-f9338608c358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:53.119301 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0128a33-89e8-4c46-aa68-f9338608c358 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:53.119583 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0128a33-89e8-4c46-aa68-f9338608c358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:53.119860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0128a33-89e8-4c46-aa68-f9338608c358 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:53.120575 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0128a33-89e8-4c46-aa68-f9338608c358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:53.122545 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0128a33-89e8-4c46-aa68-f9338608c358 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:53.214027 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 948/1885] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:11:53 2022] POST /identity/v3/users => generated 282 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:11:53.218174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902dd81-9691-463b-9a01-2e67320f7f2b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:53.230400 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902dd81-9691-463b-9a01-2e67320f7f2b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:53.239705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be479762-f785-45c3-9966-95481a3289cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:53.240401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be479762-f785-45c3-9966-95481a3289cf admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:53.240681 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be479762-f785-45c3-9966-95481a3289cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:53.240955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be479762-f785-45c3-9966-95481a3289cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:53.241948 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be479762-f785-45c3-9966-95481a3289cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:53.244006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be479762-f785-45c3-9966-95481a3289cf admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:53.256989 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 938/1886] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:53.261331 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0128a33-89e8-4c46-aa68-f9338608c358 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:53.272786 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0128a33-89e8-4c46-aa68-f9338608c358 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:53.278024 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2773ca-16c8-437b-a1dc-6ffde9974cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:53.279063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2773ca-16c8-437b-a1dc-6ffde9974cac admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:53.279353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2773ca-16c8-437b-a1dc-6ffde9974cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:53.279660 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2773ca-16c8-437b-a1dc-6ffde9974cac admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:53.280495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2773ca-16c8-437b-a1dc-6ffde9974cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:53.282122 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2773ca-16c8-437b-a1dc-6ffde9974cac admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:53.300722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 949/1887] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:53.302257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be479762-f785-45c3-9966-95481a3289cf admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:53.324776 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be479762-f785-45c3-9966-95481a3289cf admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:53.333920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6288c-9310-4832-899b-81b83554407d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:53.334981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6288c-9310-4832-899b-81b83554407d admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:53.335453 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6288c-9310-4832-899b-81b83554407d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:53.335906 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6288c-9310-4832-899b-81b83554407d admin admin] PATH_INFO: `/v3/projects/524c4bf419b347369aa147212a7dfd30/users/3b1ed251e0434d8fbffab9f343d0cb43/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:53.355769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6288c-9310-4832-899b-81b83554407d admin admin] RBAC: Authorizing `identity:create_grant(project_id=524c4bf419b347369aa147212a7dfd30, user_id=3b1ed251e0434d8fbffab9f343d0cb43, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:53.358524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f6288c-9310-4832-899b-81b83554407d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:53.438408 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 939/1888] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:11:53 2022] PUT /identity/v3/projects/524c4bf419b347369aa147212a7dfd30/users/3b1ed251e0434d8fbffab9f343d0cb43/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:53.450669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6c0fb-60cd-4e42-9139-8081ae86ee05 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:53.451460 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6c0fb-60cd-4e42-9139-8081ae86ee05 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:53.451919 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6c0fb-60cd-4e42-9139-8081ae86ee05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:53.487290 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-28e6c0fb-60cd-4e42-9139-8081ae86ee05 None None] MFA Rules not processed for user `3b1ed251e0434d8fbffab9f343d0cb43`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:53.488571 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-28e6c0fb-60cd-4e42-9139-8081ae86ee05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:53.585662 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 950/1889] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:53 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:53.590566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6288c-9310-4832-899b-81b83554407d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:53.689511 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6288c-9310-4832-899b-81b83554407d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:53.695371 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a098b87-81b6-4e09-8d2d-24e1d87c5a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:53.696196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a098b87-81b6-4e09-8d2d-24e1d87c5a09 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:53.696559 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a098b87-81b6-4e09-8d2d-24e1d87c5a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:53.699362 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a098b87-81b6-4e09-8d2d-24e1d87c5a09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:53.700136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a098b87-81b6-4e09-8d2d-24e1d87c5a09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:53.702120 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a098b87-81b6-4e09-8d2d-24e1d87c5a09 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:53.722348 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:53.846311 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 940/1890] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:11:53 2022] POST /identity/v3/projects => generated 363 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:11:53.847567 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6c0fb-60cd-4e42-9139-8081ae86ee05 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:53.860137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6c0fb-60cd-4e42-9139-8081ae86ee05 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:53.953007 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7367e02-0002-40ac-a8bf-414f9a728022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:53.955343 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7367e02-0002-40ac-a8bf-414f9a728022 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:53.955850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7367e02-0002-40ac-a8bf-414f9a728022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:53.956391 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7367e02-0002-40ac-a8bf-414f9a728022 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:53.957476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7367e02-0002-40ac-a8bf-414f9a728022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:53.960257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7367e02-0002-40ac-a8bf-414f9a728022 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:53.973720 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:54.113211 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 951/1891] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:11:53 2022] POST /identity/v3/users => generated 288 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:11:54.119039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a098b87-81b6-4e09-8d2d-24e1d87c5a09 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:54.132135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a098b87-81b6-4e09-8d2d-24e1d87c5a09 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:54.137094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c341f2e-f2a2-4efe-b4bf-2068043b61a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:54.137786 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c341f2e-f2a2-4efe-b4bf-2068043b61a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:54.138068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c341f2e-f2a2-4efe-b4bf-2068043b61a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:54.138344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c341f2e-f2a2-4efe-b4bf-2068043b61a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:54.139109 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c341f2e-f2a2-4efe-b4bf-2068043b61a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:54.140775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c341f2e-f2a2-4efe-b4bf-2068043b61a7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:54.153051 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 941/1892] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:11:54.158552 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7367e02-0002-40ac-a8bf-414f9a728022 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:54.169530 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7367e02-0002-40ac-a8bf-414f9a728022 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:54.175483 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54660c-2383-49f6-9e74-1edd833d5b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:54.176623 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54660c-2383-49f6-9e74-1edd833d5b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:54.176911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54660c-2383-49f6-9e74-1edd833d5b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:54.177175 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54660c-2383-49f6-9e74-1edd833d5b8a admin admin] PATH_INFO: `/v3/projects/53ef32ee82d44e29a8e0bd35bfaec227/users/ef09c2a7dc9d4f5a8b6dcf6b104533d2/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:54.194793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54660c-2383-49f6-9e74-1edd833d5b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ef32ee82d44e29a8e0bd35bfaec227, user_id=ef09c2a7dc9d4f5a8b6dcf6b104533d2, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:54.197634 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54660c-2383-49f6-9e74-1edd833d5b8a admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:54.461019 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 952/1893] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:11:54 2022] PUT /identity/v3/projects/53ef32ee82d44e29a8e0bd35bfaec227/users/ef09c2a7dc9d4f5a8b6dcf6b104533d2/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:54.487401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c341f2e-f2a2-4efe-b4bf-2068043b61a7 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:54.507554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c341f2e-f2a2-4efe-b4bf-2068043b61a7 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:54.603386 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f13db-03bf-4728-862e-5f8de7f7d8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:54.604293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f13db-03bf-4728-862e-5f8de7f7d8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:54.604751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f13db-03bf-4728-862e-5f8de7f7d8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:54.608305 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f13db-03bf-4728-862e-5f8de7f7d8d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:54.609303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f13db-03bf-4728-862e-5f8de7f7d8d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:54.611784 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f13db-03bf-4728-862e-5f8de7f7d8d0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:54.633890 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 942/1894] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:11:54.634566 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54660c-2383-49f6-9e74-1edd833d5b8a admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:54.650816 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54660c-2383-49f6-9e74-1edd833d5b8a admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:54.656488 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7bcf2-0361-4afc-a401-f7b42250d30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:54.658722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7bcf2-0361-4afc-a401-f7b42250d30e admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:54.659042 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7bcf2-0361-4afc-a401-f7b42250d30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:54.659317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7bcf2-0361-4afc-a401-f7b42250d30e admin admin] PATH_INFO: `/v3/projects/53ef32ee82d44e29a8e0bd35bfaec227/users/ef09c2a7dc9d4f5a8b6dcf6b104533d2/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:54.661544 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c7bcf2-0361-4afc-a401-f7b42250d30e admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ef32ee82d44e29a8e0bd35bfaec227, user_id=ef09c2a7dc9d4f5a8b6dcf6b104533d2, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:54.664461 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c7bcf2-0361-4afc-a401-f7b42250d30e admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:54.678030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 953/1895] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:11:54 2022] PUT /identity/v3/projects/53ef32ee82d44e29a8e0bd35bfaec227/users/ef09c2a7dc9d4f5a8b6dcf6b104533d2/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:54.690534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecd049-5a46-4fbc-bd09-ed8553212f05 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:54.692683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecd049-5a46-4fbc-bd09-ed8553212f05 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:54.692683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecd049-5a46-4fbc-bd09-ed8553212f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:54.742927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-90ecd049-5a46-4fbc-bd09-ed8553212f05 None None] MFA Rules not processed for user `ef09c2a7dc9d4f5a8b6dcf6b104533d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:54.749871 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-90ecd049-5a46-4fbc-bd09-ed8553212f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:54.951365 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 943/1896] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:54 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:11:55.021835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7bcf2-0361-4afc-a401-f7b42250d30e admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:55.036103 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c0c7bcf2-0361-4afc-a401-f7b42250d30e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:55.190132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7bcf2-0361-4afc-a401-f7b42250d30e admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:55.197291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e09f4-6465-4f61-ae4d-a1bbab34a77e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:55.198457 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e09f4-6465-4f61-ae4d-a1bbab34a77e service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:55.199033 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e09f4-6465-4f61-ae4d-a1bbab34a77e service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:55.199638 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e09f4-6465-4f61-ae4d-a1bbab34a77e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:55.218377 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e09f4-6465-4f61-ae4d-a1bbab34a77e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:55.221972 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e09f4-6465-4f61-ae4d-a1bbab34a77e service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:55.241008 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 954/1897] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:55 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:11:55.922008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd0cfd-8eeb-412a-a24e-ac4af03eae19 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:55.923007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd0cfd-8eeb-412a-a24e-ac4af03eae19 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:55.923007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd0cfd-8eeb-412a-a24e-ac4af03eae19 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:55.924540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 944/1898] 10.222.0.27 () {54 vars in 921 bytes} [Sat Jul 16 15:11:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:11:55.936979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91e118-429f-4559-84de-0bcdf080869e None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:55.937955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91e118-429f-4559-84de-0bcdf080869e None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:55.945539 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91e118-429f-4559-84de-0bcdf080869e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:55.971991 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-4e91e118-429f-4559-84de-0bcdf080869e None None] MFA Rules not processed for user `3a30cee766814c8685578819261705ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:55.973783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-4e91e118-429f-4559-84de-0bcdf080869e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:56.070074 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 955/1899] 10.222.0.27 () {58 vars in 1018 bytes} [Sat Jul 16 15:11:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:11:56.099123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd0cfd-8eeb-412a-a24e-ac4af03eae19 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:56.102367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-21dd0cfd-8eeb-412a-a24e-ac4af03eae19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:56.150508 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd0cfd-8eeb-412a-a24e-ac4af03eae19 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:56.150781 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91e118-429f-4559-84de-0bcdf080869e None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:56.152921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-4e91e118-429f-4559-84de-0bcdf080869e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:56.157183 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62731e9d-a165-4495-87bb-ba518854960a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a30cee766814c8685578819261705ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:56.158487 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62731e9d-a165-4495-87bb-ba518854960a service placement] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:56.158836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62731e9d-a165-4495-87bb-ba518854960a service placement] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:56.159404 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62731e9d-a165-4495-87bb-ba518854960a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:56.169559 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62731e9d-a165-4495-87bb-ba518854960a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:56.171383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62731e9d-a165-4495-87bb-ba518854960a service placement] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:56.183555 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 945/1900] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:11:56.285123 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:56.289294 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91e118-429f-4559-84de-0bcdf080869e None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:56.294526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663244b0-8c71-4a97-bca4-2e23115414fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:56.297407 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663244b0-8c71-4a97-bca4-2e23115414fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:56.297407 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663244b0-8c71-4a97-bca4-2e23115414fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:56.297407 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663244b0-8c71-4a97-bca4-2e23115414fd admin admin] PATH_INFO: `/v3/users/05da244187d54a9a8b5e4e623c3eee32` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:56.297407 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663244b0-8c71-4a97-bca4-2e23115414fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=05da244187d54a9a8b5e4e623c3eee32)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:56.299502 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663244b0-8c71-4a97-bca4-2e23115414fd admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:56.534092 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-663244b0-8c71-4a97-bca4-2e23115414fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05da244187d54a9a8b5e4e623c3eee32'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:56.637313 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62731e9d-a165-4495-87bb-ba518854960a service placement] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:56.649999 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62731e9d-a165-4495-87bb-ba518854960a service placement] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:56.654952 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0422c8-66b9-4bf2-99dd-20274b551082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:56.655858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0422c8-66b9-4bf2-99dd-20274b551082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:56.656151 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0422c8-66b9-4bf2-99dd-20274b551082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:56.656431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0422c8-66b9-4bf2-99dd-20274b551082 admin admin] PATH_INFO: `/v3/users/3b1ed251e0434d8fbffab9f343d0cb43` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:56.658679 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0422c8-66b9-4bf2-99dd-20274b551082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b1ed251e0434d8fbffab9f343d0cb43)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:56.660771 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0422c8-66b9-4bf2-99dd-20274b551082 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:56.681434 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-663244b0-8c71-4a97-bca4-2e23115414fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05da244187d54a9a8b5e4e623c3eee32'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:56.692065 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-663244b0-8c71-4a97-bca4-2e23115414fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05da244187d54a9a8b5e4e623c3eee32'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:56.707028 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-663244b0-8c71-4a97-bca4-2e23115414fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05da244187d54a9a8b5e4e623c3eee32'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:56.707802 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-663244b0-8c71-4a97-bca4-2e23115414fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05da244187d54a9a8b5e4e623c3eee32'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:56.714701 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 956/1901] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:11:56 2022] DELETE /identity/v3/users/05da244187d54a9a8b5e4e623c3eee32 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:56.872232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:INSERT=1,keystone:SELECT=92,keystone:DELETE=7 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:56.874857 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7c0422c8-66b9-4bf2-99dd-20274b551082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b1ed251e0434d8fbffab9f343d0cb43'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:56.956045 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7c0422c8-66b9-4bf2-99dd-20274b551082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b1ed251e0434d8fbffab9f343d0cb43'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:56.957041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7c0422c8-66b9-4bf2-99dd-20274b551082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b1ed251e0434d8fbffab9f343d0cb43'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:56.957851 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7c0422c8-66b9-4bf2-99dd-20274b551082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b1ed251e0434d8fbffab9f343d0cb43'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:56.966434 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-7c0422c8-66b9-4bf2-99dd-20274b551082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b1ed251e0434d8fbffab9f343d0cb43'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:56.987435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 946/1902] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:11:56 2022] DELETE /identity/v3/users/3b1ed251e0434d8fbffab9f343d0cb43 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:11:56.992681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856020ff-5e7c-4f5d-965c-350661dc0af5 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:56.992988 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856020ff-5e7c-4f5d-965c-350661dc0af5 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:56.993376 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856020ff-5e7c-4f5d-965c-350661dc0af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:57.024088 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-856020ff-5e7c-4f5d-965c-350661dc0af5 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:57.025448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-856020ff-5e7c-4f5d-965c-350661dc0af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:57.093780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0422c8-66b9-4bf2-99dd-20274b551082 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:57.097908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7c0422c8-66b9-4bf2-99dd-20274b551082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:57.100634 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 957/1903] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:57.117646 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856020ff-5e7c-4f5d-965c-350661dc0af5 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:57.119228 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-856020ff-5e7c-4f5d-965c-350661dc0af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:57.121774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0422c8-66b9-4bf2-99dd-20274b551082 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:57.128475 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dea1e-a9a2-44d1-9e90-6308b1fde438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:57.129835 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dea1e-a9a2-44d1-9e90-6308b1fde438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:57.130337 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dea1e-a9a2-44d1-9e90-6308b1fde438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:57.130745 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dea1e-a9a2-44d1-9e90-6308b1fde438 admin admin] PATH_INFO: `/v3/projects/f5e8428fd1c849a1af745df37bf095b9` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:57.132929 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8dea1e-a9a2-44d1-9e90-6308b1fde438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5e8428fd1c849a1af745df37bf095b9)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:57.135779 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8dea1e-a9a2-44d1-9e90-6308b1fde438 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:57.262013 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856020ff-5e7c-4f5d-965c-350661dc0af5 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:57.265940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c447994-e5f8-4491-9329-145399ba3745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:57.267282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c447994-e5f8-4491-9329-145399ba3745 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:57.268300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c447994-e5f8-4491-9329-145399ba3745 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:57.268588 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c447994-e5f8-4491-9329-145399ba3745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:57.279157 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c447994-e5f8-4491-9329-145399ba3745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:57.280677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c447994-e5f8-4491-9329-145399ba3745 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:57.292247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 958/1904] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:11:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:11:57.315554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fe8dea1e-a9a2-44d1-9e90-6308b1fde438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5e8428fd1c849a1af745df37bf095b9', 'request_id': 'req-fe8dea1e-a9a2-44d1-9e90-6308b1fde438'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:57.324892 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fe8dea1e-a9a2-44d1-9e90-6308b1fde438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5e8428fd1c849a1af745df37bf095b9', 'request_id': 'req-fe8dea1e-a9a2-44d1-9e90-6308b1fde438'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:57.337478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fe8dea1e-a9a2-44d1-9e90-6308b1fde438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5e8428fd1c849a1af745df37bf095b9', 'request_id': 'req-fe8dea1e-a9a2-44d1-9e90-6308b1fde438'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:57.437198 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fe8dea1e-a9a2-44d1-9e90-6308b1fde438 admin admin] The token cache is being invalidate because project f5e8428fd1c849a1af745df37bf095b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:11:57.437666 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fe8dea1e-a9a2-44d1-9e90-6308b1fde438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:57.439301 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fe8dea1e-a9a2-44d1-9e90-6308b1fde438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:57.442083 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 947/1905] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:11:57 2022] DELETE /identity/v3/projects/f5e8428fd1c849a1af745df37bf095b9 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:57.446639 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c447994-e5f8-4491-9329-145399ba3745 service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:57.459736 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c447994-e5f8-4491-9329-145399ba3745 service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:57.480877 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:57.532168 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5915976-c572-4d2d-adde-e897af1f5241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:57.533431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5915976-c572-4d2d-adde-e897af1f5241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:57.533825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5915976-c572-4d2d-adde-e897af1f5241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:57.534194 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5915976-c572-4d2d-adde-e897af1f5241 admin admin] PATH_INFO: `/v3/users/c6c879e740c740ea9091b7ab38c2d0f9` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:57.535914 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5915976-c572-4d2d-adde-e897af1f5241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6c879e740c740ea9091b7ab38c2d0f9)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:57.538695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5915976-c572-4d2d-adde-e897af1f5241 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:57.867549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c5915976-c572-4d2d-adde-e897af1f5241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6c879e740c740ea9091b7ab38c2d0f9'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:57.878162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dea1e-a9a2-44d1-9e90-6308b1fde438 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:57.880258 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-fe8dea1e-a9a2-44d1-9e90-6308b1fde438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:57.900052 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dea1e-a9a2-44d1-9e90-6308b1fde438 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:57.906538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23c88a-e92c-49bd-919c-d7f199e2f657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:57.906538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23c88a-e92c-49bd-919c-d7f199e2f657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:57.906538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23c88a-e92c-49bd-919c-d7f199e2f657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:57.906538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23c88a-e92c-49bd-919c-d7f199e2f657 admin admin] PATH_INFO: `/v3/projects/524c4bf419b347369aa147212a7dfd30` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:57.911727 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23c88a-e92c-49bd-919c-d7f199e2f657 admin admin] RBAC: Authorizing `identity:delete_project(project_id=524c4bf419b347369aa147212a7dfd30)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:57.913748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e23c88a-e92c-49bd-919c-d7f199e2f657 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:58.041838 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c5915976-c572-4d2d-adde-e897af1f5241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6c879e740c740ea9091b7ab38c2d0f9'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:58.052443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c5915976-c572-4d2d-adde-e897af1f5241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6c879e740c740ea9091b7ab38c2d0f9'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:58.066496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c5915976-c572-4d2d-adde-e897af1f5241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6c879e740c740ea9091b7ab38c2d0f9'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:58.067266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c5915976-c572-4d2d-adde-e897af1f5241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6c879e740c740ea9091b7ab38c2d0f9'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:58.074388 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 959/1906] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:11:57 2022] DELETE /identity/v3/users/c6c879e740c740ea9091b7ab38c2d0f9 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:58.102085 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1e23c88a-e92c-49bd-919c-d7f199e2f657 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '524c4bf419b347369aa147212a7dfd30', 'request_id': 'req-1e23c88a-e92c-49bd-919c-d7f199e2f657'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:58.108876 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1e23c88a-e92c-49bd-919c-d7f199e2f657 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '524c4bf419b347369aa147212a7dfd30', 'request_id': 'req-1e23c88a-e92c-49bd-919c-d7f199e2f657'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:58.118530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1e23c88a-e92c-49bd-919c-d7f199e2f657 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '524c4bf419b347369aa147212a7dfd30', 'request_id': 'req-1e23c88a-e92c-49bd-919c-d7f199e2f657'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:58.214128 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1e23c88a-e92c-49bd-919c-d7f199e2f657 admin admin] The token cache is being invalidate because project 524c4bf419b347369aa147212a7dfd30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:11:58.214707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1e23c88a-e92c-49bd-919c-d7f199e2f657 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:58.215969 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1e23c88a-e92c-49bd-919c-d7f199e2f657 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:58.221923 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 948/1907] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:11:57 2022] DELETE /identity/v3/projects/524c4bf419b347369aa147212a7dfd30 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:58.225342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5915976-c572-4d2d-adde-e897af1f5241 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:58.227529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c5915976-c572-4d2d-adde-e897af1f5241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:58.327013 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5915976-c572-4d2d-adde-e897af1f5241 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:58.376470 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:58.390695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a6c2e3-a186-41ce-a040-2314b6d2d94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:58.392134 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6c2e3-a186-41ce-a040-2314b6d2d94c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:58.392423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6c2e3-a186-41ce-a040-2314b6d2d94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:58.392702 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6c2e3-a186-41ce-a040-2314b6d2d94c admin admin] PATH_INFO: `/v3/users/ef09c2a7dc9d4f5a8b6dcf6b104533d2` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:58.394000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a6c2e3-a186-41ce-a040-2314b6d2d94c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef09c2a7dc9d4f5a8b6dcf6b104533d2)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:58.396152 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a6c2e3-a186-41ce-a040-2314b6d2d94c admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:58.414924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23c88a-e92c-49bd-919c-d7f199e2f657 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:58.415284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1e23c88a-e92c-49bd-919c-d7f199e2f657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:58.430146 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23c88a-e92c-49bd-919c-d7f199e2f657 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:58.435680 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e8bff-cca5-456f-baf3-e1b7c4948232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:58.436754 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e8bff-cca5-456f-baf3-e1b7c4948232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:58.437135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e8bff-cca5-456f-baf3-e1b7c4948232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:58.437632 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e8bff-cca5-456f-baf3-e1b7c4948232 admin admin] PATH_INFO: `/v3/projects/1c06dcabce1a4ccbb6a2d1718ef43710` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:58.440704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e8bff-cca5-456f-baf3-e1b7c4948232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c06dcabce1a4ccbb6a2d1718ef43710)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:58.444186 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e8bff-cca5-456f-baf3-e1b7c4948232 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:58.477395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=79,keystone:INSERT=3,keystone:DELETE=18 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:58.633839 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-538e8bff-cca5-456f-baf3-e1b7c4948232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c06dcabce1a4ccbb6a2d1718ef43710', 'request_id': 'req-538e8bff-cca5-456f-baf3-e1b7c4948232'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:58.640580 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c9a6c2e3-a186-41ce-a040-2314b6d2d94c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef09c2a7dc9d4f5a8b6dcf6b104533d2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:58.641355 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-538e8bff-cca5-456f-baf3-e1b7c4948232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c06dcabce1a4ccbb6a2d1718ef43710', 'request_id': 'req-538e8bff-cca5-456f-baf3-e1b7c4948232'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:58.655507 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-538e8bff-cca5-456f-baf3-e1b7c4948232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c06dcabce1a4ccbb6a2d1718ef43710', 'request_id': 'req-538e8bff-cca5-456f-baf3-e1b7c4948232'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:58.761170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c9a6c2e3-a186-41ce-a040-2314b6d2d94c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef09c2a7dc9d4f5a8b6dcf6b104533d2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:58.772585 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c9a6c2e3-a186-41ce-a040-2314b6d2d94c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef09c2a7dc9d4f5a8b6dcf6b104533d2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:58.789747 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c9a6c2e3-a186-41ce-a040-2314b6d2d94c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef09c2a7dc9d4f5a8b6dcf6b104533d2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:58.797086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c9a6c2e3-a186-41ce-a040-2314b6d2d94c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef09c2a7dc9d4f5a8b6dcf6b104533d2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:58.801410 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 960/1908] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:11:58 2022] DELETE /identity/v3/users/ef09c2a7dc9d4f5a8b6dcf6b104533d2 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:11:58.856384 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-538e8bff-cca5-456f-baf3-e1b7c4948232 admin admin] The token cache is being invalidate because project 1c06dcabce1a4ccbb6a2d1718ef43710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:11:58.856849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-538e8bff-cca5-456f-baf3-e1b7c4948232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:58.861031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-538e8bff-cca5-456f-baf3-e1b7c4948232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:58.861031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 949/1909] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:11:58 2022] DELETE /identity/v3/projects/1c06dcabce1a4ccbb6a2d1718ef43710 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:11:58.865957 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae951b-45c9-4900-9781-3229ab0e6d62 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:58.866393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae951b-45c9-4900-9781-3229ab0e6d62 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:58.866723 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae951b-45c9-4900-9781-3229ab0e6d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:58.918218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-3aae951b-45c9-4900-9781-3229ab0e6d62 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:58.920678 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3aae951b-45c9-4900-9781-3229ab0e6d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:59.023214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 961/1910] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:11:59.034200 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35796a12-6e15-4eef-92da-6385f3368b2b None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:59.034588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35796a12-6e15-4eef-92da-6385f3368b2b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:59.035027 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35796a12-6e15-4eef-92da-6385f3368b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:59.060329 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-35796a12-6e15-4eef-92da-6385f3368b2b None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:59.067391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-35796a12-6e15-4eef-92da-6385f3368b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:59.076635 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 950/1911] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:11:59.083264 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae951b-45c9-4900-9781-3229ab0e6d62 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:59.103817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae951b-45c9-4900-9781-3229ab0e6d62 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:59.112728 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9e82d-ebdf-4f68-acbe-f71f3218057a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:59.113525 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9e82d-ebdf-4f68-acbe-f71f3218057a admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:59.116728 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9e82d-ebdf-4f68-acbe-f71f3218057a admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:59.116728 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9e82d-ebdf-4f68-acbe-f71f3218057a admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:59.116728 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9e82d-ebdf-4f68-acbe-f71f3218057a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:11:59.116728 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9e82d-ebdf-4f68-acbe-f71f3218057a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:59.117276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9e82d-ebdf-4f68-acbe-f71f3218057a admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:59.131650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 962/1912] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:11:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:11:59.136058 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35796a12-6e15-4eef-92da-6385f3368b2b None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:59.147516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35796a12-6e15-4eef-92da-6385f3368b2b None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:59.153964 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ee0ea-d909-451d-bd28-5ef3a928ceca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:59.154942 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ee0ea-d909-451d-bd28-5ef3a928ceca admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:59.155538 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ee0ea-d909-451d-bd28-5ef3a928ceca admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:59.156088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ee0ea-d909-451d-bd28-5ef3a928ceca admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:59.158618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ee0ea-d909-451d-bd28-5ef3a928ceca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:59.163051 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ee0ea-d909-451d-bd28-5ef3a928ceca admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:59.290950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 951/1913] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:11:59 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:11:59.295076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9e82d-ebdf-4f68-acbe-f71f3218057a admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:59.314414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9e82d-ebdf-4f68-acbe-f71f3218057a admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:59.400328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:11:59.410144 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ee0ea-d909-451d-bd28-5ef3a928ceca admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:59.413382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-164ee0ea-d909-451d-bd28-5ef3a928ceca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:59.415230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581b02e-e6e8-4ed4-a852-7358591fd143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:59.416369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581b02e-e6e8-4ed4-a852-7358591fd143 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:59.416874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581b02e-e6e8-4ed4-a852-7358591fd143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:59.417324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581b02e-e6e8-4ed4-a852-7358591fd143 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:59.418731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581b02e-e6e8-4ed4-a852-7358591fd143 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:59.421847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581b02e-e6e8-4ed4-a852-7358591fd143 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:59.436087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ee0ea-d909-451d-bd28-5ef3a928ceca admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:59.443262 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789de64-7601-4c44-91a7-51f9c1d78ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:59.445159 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789de64-7601-4c44-91a7-51f9c1d78ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:59.445376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789de64-7601-4c44-91a7-51f9c1d78ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:59.445699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789de64-7601-4c44-91a7-51f9c1d78ce5 admin admin] PATH_INFO: `/v3/projects/53ef32ee82d44e29a8e0bd35bfaec227` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:59.447286 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789de64-7601-4c44-91a7-51f9c1d78ce5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53ef32ee82d44e29a8e0bd35bfaec227)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:59.451348 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789de64-7601-4c44-91a7-51f9c1d78ce5 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:59.516364 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 963/1914] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:11:59 2022] POST /identity/v3/users => generated 292 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:11:59.520909 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581b02e-e6e8-4ed4-a852-7358591fd143 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:59.531451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581b02e-e6e8-4ed4-a852-7358591fd143 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:59.536864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17129a72-7f7a-48d4-bb51-26fae9857e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:59.537541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17129a72-7f7a-48d4-bb51-26fae9857e74 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:59.537828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17129a72-7f7a-48d4-bb51-26fae9857e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:59.538108 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17129a72-7f7a-48d4-bb51-26fae9857e74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:59.538842 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17129a72-7f7a-48d4-bb51-26fae9857e74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:59.540482 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17129a72-7f7a-48d4-bb51-26fae9857e74 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:59.552443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 964/1915] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:11:59.556471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17129a72-7f7a-48d4-bb51-26fae9857e74 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:59.567759 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17129a72-7f7a-48d4-bb51-26fae9857e74 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:59.573942 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82d144-b6f4-4d67-9d99-a497af35d3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:59.575026 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82d144-b6f4-4d67-9d99-a497af35d3fd admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:59.575602 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82d144-b6f4-4d67-9d99-a497af35d3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:59.575946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82d144-b6f4-4d67-9d99-a497af35d3fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:59.576848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82d144-b6f4-4d67-9d99-a497af35d3fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:59.579788 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82d144-b6f4-4d67-9d99-a497af35d3fd admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:59.596246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 965/1916] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:11:59.599763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82d144-b6f4-4d67-9d99-a497af35d3fd admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:11:59.604366 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3789de64-7601-4c44-91a7-51f9c1d78ce5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53ef32ee82d44e29a8e0bd35bfaec227', 'request_id': 'req-3789de64-7601-4c44-91a7-51f9c1d78ce5'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:59.611378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82d144-b6f4-4d67-9d99-a497af35d3fd admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:11:59.611679 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3789de64-7601-4c44-91a7-51f9c1d78ce5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53ef32ee82d44e29a8e0bd35bfaec227', 'request_id': 'req-3789de64-7601-4c44-91a7-51f9c1d78ce5'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:59.618813 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3789de64-7601-4c44-91a7-51f9c1d78ce5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53ef32ee82d44e29a8e0bd35bfaec227', 'request_id': 'req-3789de64-7601-4c44-91a7-51f9c1d78ce5'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:59.681503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c6a3b-15eb-468a-bb28-b77e36153543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:11:59.688556 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c6a3b-15eb-468a-bb28-b77e36153543 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:59.688556 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c6a3b-15eb-468a-bb28-b77e36153543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:59.688556 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c6a3b-15eb-468a-bb28-b77e36153543 admin admin] PATH_INFO: `/v3/projects/f70ea05421084253871d7968389c9533/users/989ba287fb3c478c8e275474e6dc878f/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:59.701660 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c6a3b-15eb-468a-bb28-b77e36153543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f70ea05421084253871d7968389c9533, user_id=989ba287fb3c478c8e275474e6dc878f, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:11:59.705779 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c6a3b-15eb-468a-bb28-b77e36153543 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:11:59.754718 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3789de64-7601-4c44-91a7-51f9c1d78ce5 admin admin] The token cache is being invalidate because project 53ef32ee82d44e29a8e0bd35bfaec227 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:11:59.755396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3789de64-7601-4c44-91a7-51f9c1d78ce5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:59.756931 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3789de64-7601-4c44-91a7-51f9c1d78ce5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:11:59.762509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 952/1917] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:11:59 2022] DELETE /identity/v3/projects/53ef32ee82d44e29a8e0bd35bfaec227 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:11:59.772901 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963b8cf-da59-4edf-a59b-fad9d154e866 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:59.773542 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963b8cf-da59-4edf-a59b-fad9d154e866 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:59.773915 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963b8cf-da59-4edf-a59b-fad9d154e866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:59.799493 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d963b8cf-da59-4edf-a59b-fad9d154e866 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:59.801103 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d963b8cf-da59-4edf-a59b-fad9d154e866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:59.825813 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 953/1918] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:11:59.832432 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beead8db-7a6d-4997-8f8b-adfbbb34c68d None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:59.832921 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beead8db-7a6d-4997-8f8b-adfbbb34c68d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:59.833371 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beead8db-7a6d-4997-8f8b-adfbbb34c68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:59.862826 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 966/1919] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:11:59 2022] PUT /identity/v3/projects/f70ea05421084253871d7968389c9533/users/989ba287fb3c478c8e275474e6dc878f/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:11:59.874345 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9b810-af68-44f9-a94f-2bf7ddb94444 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:11:59.887997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9b810-af68-44f9-a94f-2bf7ddb94444 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:11:59.888662 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9b810-af68-44f9-a94f-2bf7ddb94444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:11:59.899806 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-beead8db-7a6d-4997-8f8b-adfbbb34c68d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:59.903597 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-beead8db-7a6d-4997-8f8b-adfbbb34c68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:11:59.960852 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-91c9b810-af68-44f9-a94f-2bf7ddb94444 None None] MFA Rules not processed for user `989ba287fb3c478c8e275474e6dc878f`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:11:59.962339 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-91c9b810-af68-44f9-a94f-2bf7ddb94444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:00.029361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:00.062034 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 954/1920] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:12:00.064899 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beead8db-7a6d-4997-8f8b-adfbbb34c68d None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:00.072491 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 967/1921] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:11:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:12:00.080780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beead8db-7a6d-4997-8f8b-adfbbb34c68d None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:00.085846 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ffe4b-3a60-43c1-8554-fb6521a42804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:00.086936 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ffe4b-3a60-43c1-8554-fb6521a42804 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:00.086936 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ffe4b-3a60-43c1-8554-fb6521a42804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:00.087151 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ffe4b-3a60-43c1-8554-fb6521a42804 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:00.087869 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ffe4b-3a60-43c1-8554-fb6521a42804 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:12:00.088210 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ffe4b-3a60-43c1-8554-fb6521a42804 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:00.090121 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ffe4b-3a60-43c1-8554-fb6521a42804 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:00.112380 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 955/1922] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:12:00.117908 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9b810-af68-44f9-a94f-2bf7ddb94444 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:00.137223 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9b810-af68-44f9-a94f-2bf7ddb94444 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:00.149591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0072f-deff-49ff-9714-83a86996f00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:00.150770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0072f-deff-49ff-9714-83a86996f00c admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:00.151320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0072f-deff-49ff-9714-83a86996f00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:00.151815 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0072f-deff-49ff-9714-83a86996f00c admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:00.153185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba0072f-deff-49ff-9714-83a86996f00c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:00.158015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba0072f-deff-49ff-9714-83a86996f00c admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:00.222244 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ffe4b-3a60-43c1-8554-fb6521a42804 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:00.224711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-212ffe4b-3a60-43c1-8554-fb6521a42804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:00.308779 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 968/1923] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:12:00 2022] POST /identity/v3/projects => generated 373 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:00.316072 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0072f-deff-49ff-9714-83a86996f00c admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:00.337493 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ffe4b-3a60-43c1-8554-fb6521a42804 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:00.417537 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a099a-ca26-42fe-b85c-56cf75fbe0ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:00.418108 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a099a-ca26-42fe-b85c-56cf75fbe0ff service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:00.418396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a099a-ca26-42fe-b85c-56cf75fbe0ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:00.418672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a099a-ca26-42fe-b85c-56cf75fbe0ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:00.433697 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0072f-deff-49ff-9714-83a86996f00c admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:00.433697 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df2130-b234-4630-8c3e-af0165cae911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:00.434511 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df2130-b234-4630-8c3e-af0165cae911 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:00.434811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df2130-b234-4630-8c3e-af0165cae911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:00.435290 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df2130-b234-4630-8c3e-af0165cae911 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:00.435921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df2130-b234-4630-8c3e-af0165cae911 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:00.437974 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df2130-b234-4630-8c3e-af0165cae911 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:00.488730 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a099a-ca26-42fe-b85c-56cf75fbe0ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:00.490275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a099a-ca26-42fe-b85c-56cf75fbe0ff service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:00.500411 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 956/1924] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:00.560317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 969/1925] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:12:00 2022] POST /identity/v3/users => generated 287 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:00.565895 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a099a-ca26-42fe-b85c-56cf75fbe0ff service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:00.604528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a099a-ca26-42fe-b85c-56cf75fbe0ff service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:00.610729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6cef3-a85c-4e5e-840f-f0b67a1fb9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:00.611924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6cef3-a85c-4e5e-840f-f0b67a1fb9fe admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:00.612406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6cef3-a85c-4e5e-840f-f0b67a1fb9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:00.612577 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6cef3-a85c-4e5e-840f-f0b67a1fb9fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:00.613403 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6cef3-a85c-4e5e-840f-f0b67a1fb9fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:00.615284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6cef3-a85c-4e5e-840f-f0b67a1fb9fe admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:00.628450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 957/1926] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:00.633036 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df2130-b234-4630-8c3e-af0165cae911 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:00.646923 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df2130-b234-4630-8c3e-af0165cae911 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:00.653320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1db9f3-d960-48cc-82d9-aec944e0e158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:00.654157 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1db9f3-d960-48cc-82d9-aec944e0e158 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:00.654542 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1db9f3-d960-48cc-82d9-aec944e0e158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:00.654940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1db9f3-d960-48cc-82d9-aec944e0e158 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:00.655792 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1db9f3-d960-48cc-82d9-aec944e0e158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:00.658607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1db9f3-d960-48cc-82d9-aec944e0e158 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:00.674306 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 970/1927] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:12:00.685137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6cef3-a85c-4e5e-840f-f0b67a1fb9fe admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:00.697414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6cef3-a85c-4e5e-840f-f0b67a1fb9fe admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:00.703589 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62045c94-60c7-4da4-9867-65da42fbeaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:00.705158 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62045c94-60c7-4da4-9867-65da42fbeaa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:00.705511 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62045c94-60c7-4da4-9867-65da42fbeaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:00.705684 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62045c94-60c7-4da4-9867-65da42fbeaa7 admin admin] PATH_INFO: `/v3/projects/eeca2b690f9e4d7bb07f177ca9b067ff/users/f715168afae2450da11e6c8a32511bf0/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:00.729376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62045c94-60c7-4da4-9867-65da42fbeaa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeca2b690f9e4d7bb07f177ca9b067ff, user_id=f715168afae2450da11e6c8a32511bf0, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:00.732448 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62045c94-60c7-4da4-9867-65da42fbeaa7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:00.775070 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:00.828643 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 958/1928] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:00 2022] PUT /identity/v3/projects/eeca2b690f9e4d7bb07f177ca9b067ff/users/f715168afae2450da11e6c8a32511bf0/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:00.850785 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c92783-c8c2-4806-8c5c-51c345e6c274 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:00.851282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c92783-c8c2-4806-8c5c-51c345e6c274 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:00.851624 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c92783-c8c2-4806-8c5c-51c345e6c274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:00.885315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-b7c92783-c8c2-4806-8c5c-51c345e6c274 None None] MFA Rules not processed for user `f715168afae2450da11e6c8a32511bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:00.888786 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b7c92783-c8c2-4806-8c5c-51c345e6c274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:00.996275 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 971/1929] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:01.003601 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62045c94-60c7-4da4-9867-65da42fbeaa7 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:01.087050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62045c94-60c7-4da4-9867-65da42fbeaa7 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:01.093102 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab550839-394f-42f7-b496-b9737c46f244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:01.093943 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab550839-394f-42f7-b496-b9737c46f244 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:01.094226 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab550839-394f-42f7-b496-b9737c46f244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:01.094537 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab550839-394f-42f7-b496-b9737c46f244 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:01.095358 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab550839-394f-42f7-b496-b9737c46f244 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:01.097380 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab550839-394f-42f7-b496-b9737c46f244 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:01.221499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 959/1930] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:12:01 2022] POST /identity/v3/projects => generated 375 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:12:01.223184 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c92783-c8c2-4806-8c5c-51c345e6c274 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:01.237325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c92783-c8c2-4806-8c5c-51c345e6c274 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:01.390664 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69621b98-47b1-4bab-bfba-b56beb2e6b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:01.392311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69621b98-47b1-4bab-bfba-b56beb2e6b6f admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:01.392794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69621b98-47b1-4bab-bfba-b56beb2e6b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:01.393260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69621b98-47b1-4bab-bfba-b56beb2e6b6f admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:01.394173 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69621b98-47b1-4bab-bfba-b56beb2e6b6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:01.396451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69621b98-47b1-4bab-bfba-b56beb2e6b6f admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:01.441995 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:01.519751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 972/1931] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:12:01 2022] POST /identity/v3/users => generated 294 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:12:01.523850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab550839-394f-42f7-b496-b9737c46f244 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:01.538424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab550839-394f-42f7-b496-b9737c46f244 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:01.546265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85536175-57cb-4004-ab3f-e42ccdbb2410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:01.547396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85536175-57cb-4004-ab3f-e42ccdbb2410 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:01.548067 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85536175-57cb-4004-ab3f-e42ccdbb2410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:01.548472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85536175-57cb-4004-ab3f-e42ccdbb2410 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:01.554529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85536175-57cb-4004-ab3f-e42ccdbb2410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:01.554529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85536175-57cb-4004-ab3f-e42ccdbb2410 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:01.567567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 960/1932] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:12:01.573309 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69621b98-47b1-4bab-bfba-b56beb2e6b6f admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:01.584927 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69621b98-47b1-4bab-bfba-b56beb2e6b6f admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:01.590011 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d16c0-498c-42a4-8cd1-9876cdb3753d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:01.591084 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d16c0-498c-42a4-8cd1-9876cdb3753d admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:01.591452 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d16c0-498c-42a4-8cd1-9876cdb3753d admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:01.591798 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d16c0-498c-42a4-8cd1-9876cdb3753d admin admin] PATH_INFO: `/v3/projects/b07da26c1e0a4bdb9b48a96752d67552/users/ae43573342734a5c83f05dfaa58691d8/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:01.612123 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d16c0-498c-42a4-8cd1-9876cdb3753d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b07da26c1e0a4bdb9b48a96752d67552, user_id=ae43573342734a5c83f05dfaa58691d8, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:01.614990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d16c0-498c-42a4-8cd1-9876cdb3753d admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:01.647665 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:01.687651 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 973/1933] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:01 2022] PUT /identity/v3/projects/b07da26c1e0a4bdb9b48a96752d67552/users/ae43573342734a5c83f05dfaa58691d8/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:12:01.692315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85536175-57cb-4004-ab3f-e42ccdbb2410 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:01.702827 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85536175-57cb-4004-ab3f-e42ccdbb2410 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:01.764134 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391658b-0ada-4b64-b4fd-e787c529f242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:01.764956 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391658b-0ada-4b64-b4fd-e787c529f242 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:01.765308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391658b-0ada-4b64-b4fd-e787c529f242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:01.765651 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391658b-0ada-4b64-b4fd-e787c529f242 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:01.767405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a391658b-0ada-4b64-b4fd-e787c529f242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:01.769183 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a391658b-0ada-4b64-b4fd-e787c529f242 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:01.784027 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 961/1934] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:01.785203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d16c0-498c-42a4-8cd1-9876cdb3753d admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:01.796632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d16c0-498c-42a4-8cd1-9876cdb3753d admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:01.802686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c6a26-ac3b-4ae7-99fe-c047319110dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:01.803850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c6a26-ac3b-4ae7-99fe-c047319110dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:01.808281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c6a26-ac3b-4ae7-99fe-c047319110dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:01.808281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c6a26-ac3b-4ae7-99fe-c047319110dd admin admin] PATH_INFO: `/v3/projects/b07da26c1e0a4bdb9b48a96752d67552/users/ae43573342734a5c83f05dfaa58691d8/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:01.808281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c6a26-ac3b-4ae7-99fe-c047319110dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b07da26c1e0a4bdb9b48a96752d67552, user_id=ae43573342734a5c83f05dfaa58691d8, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:01.809822 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c6a26-ac3b-4ae7-99fe-c047319110dd admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:01.822402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 974/1935] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:01 2022] PUT /identity/v3/projects/b07da26c1e0a4bdb9b48a96752d67552/users/ae43573342734a5c83f05dfaa58691d8/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:01.834658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a37ad4-6eb4-4ea6-82c9-d780984d187f None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:01.837035 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a37ad4-6eb4-4ea6-82c9-d780984d187f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:01.837035 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a37ad4-6eb4-4ea6-82c9-d780984d187f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:01.865531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c2a37ad4-6eb4-4ea6-82c9-d780984d187f None None] MFA Rules not processed for user `ae43573342734a5c83f05dfaa58691d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:01.866917 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c2a37ad4-6eb4-4ea6-82c9-d780984d187f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:01.960328 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 962/1936] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:12:01.971580 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c6a26-ac3b-4ae7-99fe-c047319110dd admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:02.039713 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c6a26-ac3b-4ae7-99fe-c047319110dd admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:02.044027 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2420f-0d6e-45a9-adb8-770667746b65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:02.044696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2420f-0d6e-45a9-adb8-770667746b65 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:02.045100 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2420f-0d6e-45a9-adb8-770667746b65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:02.045383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2420f-0d6e-45a9-adb8-770667746b65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:02.115554 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2420f-0d6e-45a9-adb8-770667746b65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:02.117043 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2420f-0d6e-45a9-adb8-770667746b65 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:02.128140 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 975/1937] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:11.105300 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a37ad4-6eb4-4ea6-82c9-d780984d187f None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:11.145105 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:11.203315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a37ad4-6eb4-4ea6-82c9-d780984d187f None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:11.209907 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46a942c-026d-41fd-aabe-2ad5781ea15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:11.210981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a942c-026d-41fd-aabe-2ad5781ea15d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:11.211283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a942c-026d-41fd-aabe-2ad5781ea15d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:11.211549 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a942c-026d-41fd-aabe-2ad5781ea15d admin admin] PATH_INFO: `/v3/users/f715168afae2450da11e6c8a32511bf0` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:11.212916 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46a942c-026d-41fd-aabe-2ad5781ea15d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f715168afae2450da11e6c8a32511bf0)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:11.214929 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46a942c-026d-41fd-aabe-2ad5781ea15d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:11.533396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f46a942c-026d-41fd-aabe-2ad5781ea15d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f715168afae2450da11e6c8a32511bf0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:11.613080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f46a942c-026d-41fd-aabe-2ad5781ea15d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f715168afae2450da11e6c8a32511bf0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:11.613713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f46a942c-026d-41fd-aabe-2ad5781ea15d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f715168afae2450da11e6c8a32511bf0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:11.614343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f46a942c-026d-41fd-aabe-2ad5781ea15d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f715168afae2450da11e6c8a32511bf0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:11.630172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f46a942c-026d-41fd-aabe-2ad5781ea15d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f715168afae2450da11e6c8a32511bf0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:11.663951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 963/1938] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:12:11 2022] DELETE /identity/v3/users/f715168afae2450da11e6c8a32511bf0 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:11.669933 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad331d-250f-476d-b1d0-9d97e54ca955 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:11.670510 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad331d-250f-476d-b1d0-9d97e54ca955 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:11.670510 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad331d-250f-476d-b1d0-9d97e54ca955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:11.691163 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-12ad331d-250f-476d-b1d0-9d97e54ca955 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:11.692709 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-12ad331d-250f-476d-b1d0-9d97e54ca955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:11.826964 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 976/1939] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:11.836911 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46a942c-026d-41fd-aabe-2ad5781ea15d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:11.839913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f46a942c-026d-41fd-aabe-2ad5781ea15d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:11.933880 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46a942c-026d-41fd-aabe-2ad5781ea15d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:11.941751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c82e15-e97b-403d-82cb-9e9974319c64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:11.943752 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c82e15-e97b-403d-82cb-9e9974319c64 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:11.952733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c82e15-e97b-403d-82cb-9e9974319c64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:11.952733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c82e15-e97b-403d-82cb-9e9974319c64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:11.958724 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c82e15-e97b-403d-82cb-9e9974319c64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:11.960419 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c82e15-e97b-403d-82cb-9e9974319c64 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:11.975918 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 964/1940] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:12.508547 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad331d-250f-476d-b1d0-9d97e54ca955 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:12.508547 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-12ad331d-250f-476d-b1d0-9d97e54ca955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:12.526649 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad331d-250f-476d-b1d0-9d97e54ca955 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:12.543688 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395b284-5e54-448f-8520-6815a2a62d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:12.544620 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395b284-5e54-448f-8520-6815a2a62d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:12.544918 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395b284-5e54-448f-8520-6815a2a62d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:12.545196 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395b284-5e54-448f-8520-6815a2a62d23 admin admin] PATH_INFO: `/v3/projects/eeca2b690f9e4d7bb07f177ca9b067ff` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:12.546591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395b284-5e54-448f-8520-6815a2a62d23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeca2b690f9e4d7bb07f177ca9b067ff)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:12.549249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6395b284-5e54-448f-8520-6815a2a62d23 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:12.577220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:12.692644 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-6395b284-5e54-448f-8520-6815a2a62d23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeca2b690f9e4d7bb07f177ca9b067ff', 'request_id': 'req-6395b284-5e54-448f-8520-6815a2a62d23'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:12.701370 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-6395b284-5e54-448f-8520-6815a2a62d23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeca2b690f9e4d7bb07f177ca9b067ff', 'request_id': 'req-6395b284-5e54-448f-8520-6815a2a62d23'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:12.716176 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-6395b284-5e54-448f-8520-6815a2a62d23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeca2b690f9e4d7bb07f177ca9b067ff', 'request_id': 'req-6395b284-5e54-448f-8520-6815a2a62d23'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:12.811102 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-6395b284-5e54-448f-8520-6815a2a62d23 admin admin] The token cache is being invalidate because project eeca2b690f9e4d7bb07f177ca9b067ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:12:12.811684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-6395b284-5e54-448f-8520-6815a2a62d23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:12.813612 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-6395b284-5e54-448f-8520-6815a2a62d23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:12.816425 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 977/1941] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:12:12 2022] DELETE /identity/v3/projects/eeca2b690f9e4d7bb07f177ca9b067ff => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:12.821350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c82e15-e97b-403d-82cb-9e9974319c64 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:12.840107 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c82e15-e97b-403d-82cb-9e9974319c64 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:12.853444 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395b284-5e54-448f-8520-6815a2a62d23 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:12.855699 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-6395b284-5e54-448f-8520-6815a2a62d23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:12.949998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:12.958821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6395b284-5e54-448f-8520-6815a2a62d23 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:12.966500 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8f323-ce69-4b19-b900-a25972caf92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:12.967842 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8f323-ce69-4b19-b900-a25972caf92e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:12.969623 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8f323-ce69-4b19-b900-a25972caf92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:12.969623 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8f323-ce69-4b19-b900-a25972caf92e admin admin] PATH_INFO: `/v3/users/989ba287fb3c478c8e275474e6dc878f` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:12.971226 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8f323-ce69-4b19-b900-a25972caf92e admin admin] RBAC: Authorizing `identity:delete_user(user_id=989ba287fb3c478c8e275474e6dc878f)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:12.973345 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8f323-ce69-4b19-b900-a25972caf92e admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:12.994210 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd8242-00f3-441b-a05c-803f60adcd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:12.995608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd8242-00f3-441b-a05c-803f60adcd7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:12.996061 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd8242-00f3-441b-a05c-803f60adcd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:12.996480 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd8242-00f3-441b-a05c-803f60adcd7b admin admin] PATH_INFO: `/v3/users/ae43573342734a5c83f05dfaa58691d8` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:12.998480 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd8242-00f3-441b-a05c-803f60adcd7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae43573342734a5c83f05dfaa58691d8)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:13.001762 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd8242-00f3-441b-a05c-803f60adcd7b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:13.259486 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c7c8f323-ce69-4b19-b900-a25972caf92e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '989ba287fb3c478c8e275474e6dc878f'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:13.261075 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a7dd8242-00f3-441b-a05c-803f60adcd7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae43573342734a5c83f05dfaa58691d8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:13.383900 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a7dd8242-00f3-441b-a05c-803f60adcd7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae43573342734a5c83f05dfaa58691d8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:13.384844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a7dd8242-00f3-441b-a05c-803f60adcd7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae43573342734a5c83f05dfaa58691d8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:13.385707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a7dd8242-00f3-441b-a05c-803f60adcd7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae43573342734a5c83f05dfaa58691d8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:13.396498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a7dd8242-00f3-441b-a05c-803f60adcd7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae43573342734a5c83f05dfaa58691d8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:13.418384 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 965/1942] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:12:12 2022] DELETE /identity/v3/users/ae43573342734a5c83f05dfaa58691d8 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:12:13.476251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd8242-00f3-441b-a05c-803f60adcd7b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:13.478078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a7dd8242-00f3-441b-a05c-803f60adcd7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:13.488789 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c7c8f323-ce69-4b19-b900-a25972caf92e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '989ba287fb3c478c8e275474e6dc878f'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:13.504986 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c7c8f323-ce69-4b19-b900-a25972caf92e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '989ba287fb3c478c8e275474e6dc878f'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:13.533496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c7c8f323-ce69-4b19-b900-a25972caf92e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '989ba287fb3c478c8e275474e6dc878f'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:13.534474 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c7c8f323-ce69-4b19-b900-a25972caf92e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '989ba287fb3c478c8e275474e6dc878f'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:13.545620 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 978/1943] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:12:12 2022] DELETE /identity/v3/users/989ba287fb3c478c8e275474e6dc878f => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:12:13.552507 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b69fd3-4b85-4327-a8ab-327faa90a9e2 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:13.552964 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b69fd3-4b85-4327-a8ab-327faa90a9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:13.553420 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b69fd3-4b85-4327-a8ab-327faa90a9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:13.581978 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-d7b69fd3-4b85-4327-a8ab-327faa90a9e2 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:13.583510 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-d7b69fd3-4b85-4327-a8ab-327faa90a9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:13.593712 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd8242-00f3-441b-a05c-803f60adcd7b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:13.611751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:13.655120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 979/1944] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:12:13.665483 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b69fd3-4b85-4327-a8ab-327faa90a9e2 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:13.669874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-d7b69fd3-4b85-4327-a8ab-327faa90a9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:13.671736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d95a78-c41f-4edb-a9c1-db1719751dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:13.673020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d95a78-c41f-4edb-a9c1-db1719751dff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:13.673419 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d95a78-c41f-4edb-a9c1-db1719751dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:13.673832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d95a78-c41f-4edb-a9c1-db1719751dff admin admin] PATH_INFO: `/v3/users/9ebbef1f078044bfb8ebb741d2667da7` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:13.675395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d95a78-c41f-4edb-a9c1-db1719751dff admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ebbef1f078044bfb8ebb741d2667da7)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:13.677662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d95a78-c41f-4edb-a9c1-db1719751dff admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:13.684786 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:13.799570 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b69fd3-4b85-4327-a8ab-327faa90a9e2 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:13.800466 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a42a7-2040-4738-8155-c8136c535bfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:13.802188 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a42a7-2040-4738-8155-c8136c535bfe service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:13.802918 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a42a7-2040-4738-8155-c8136c535bfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:13.803677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a42a7-2040-4738-8155-c8136c535bfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:13.825479 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a42a7-2040-4738-8155-c8136c535bfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:13.828982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a42a7-2040-4738-8155-c8136c535bfe service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:13.849712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 980/1945] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:13.980373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-02d95a78-c41f-4edb-a9c1-db1719751dff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ebbef1f078044bfb8ebb741d2667da7'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:14.059475 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-02d95a78-c41f-4edb-a9c1-db1719751dff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ebbef1f078044bfb8ebb741d2667da7'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:14.060276 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-02d95a78-c41f-4edb-a9c1-db1719751dff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ebbef1f078044bfb8ebb741d2667da7'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:14.061188 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-02d95a78-c41f-4edb-a9c1-db1719751dff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ebbef1f078044bfb8ebb741d2667da7'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:14.071941 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-02d95a78-c41f-4edb-a9c1-db1719751dff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ebbef1f078044bfb8ebb741d2667da7'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:14.088117 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a42a7-2040-4738-8155-c8136c535bfe service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:14.090294 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7a1a42a7-2040-4738-8155-c8136c535bfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:14.096221 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 966/1946] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:12:13 2022] DELETE /identity/v3/users/9ebbef1f078044bfb8ebb741d2667da7 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:12:14.101149 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53faed-d46c-4d96-a586-09c6a0de4c43 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:14.103780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53faed-d46c-4d96-a586-09c6a0de4c43 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:14.103780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53faed-d46c-4d96-a586-09c6a0de4c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:14.116056 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a42a7-2040-4738-8155-c8136c535bfe service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:14.152065 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3e53faed-d46c-4d96-a586-09c6a0de4c43 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:14.154459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3e53faed-d46c-4d96-a586-09c6a0de4c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:14.214964 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f10453-0c77-4c18-984a-d780d029d010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:14.216996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f10453-0c77-4c18-984a-d780d029d010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:14.217280 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f10453-0c77-4c18-984a-d780d029d010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:14.217852 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f10453-0c77-4c18-984a-d780d029d010 admin admin] PATH_INFO: `/v3/projects/b07da26c1e0a4bdb9b48a96752d67552` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:14.220751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f10453-0c77-4c18-984a-d780d029d010 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b07da26c1e0a4bdb9b48a96752d67552)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:14.224666 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f10453-0c77-4c18-984a-d780d029d010 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:14.245216 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 967/1947] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:12:14.257042 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53faed-d46c-4d96-a586-09c6a0de4c43 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:14.260050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3e53faed-d46c-4d96-a586-09c6a0de4c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:14.372949 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53faed-d46c-4d96-a586-09c6a0de4c43 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:14.396512 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29327dd-bccb-4d0c-9b89-4ba6915b266e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:14.401924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29327dd-bccb-4d0c-9b89-4ba6915b266e service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:14.402410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29327dd-bccb-4d0c-9b89-4ba6915b266e service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:14.403290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29327dd-bccb-4d0c-9b89-4ba6915b266e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:14.429343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29327dd-bccb-4d0c-9b89-4ba6915b266e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:14.437870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29327dd-bccb-4d0c-9b89-4ba6915b266e service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:14.444201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-70f10453-0c77-4c18-984a-d780d029d010 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b07da26c1e0a4bdb9b48a96752d67552', 'request_id': 'req-70f10453-0c77-4c18-984a-d780d029d010'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:14.464312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-70f10453-0c77-4c18-984a-d780d029d010 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b07da26c1e0a4bdb9b48a96752d67552', 'request_id': 'req-70f10453-0c77-4c18-984a-d780d029d010'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:14.469687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 968/1948] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:14.485233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-70f10453-0c77-4c18-984a-d780d029d010 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b07da26c1e0a4bdb9b48a96752d67552', 'request_id': 'req-70f10453-0c77-4c18-984a-d780d029d010'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:14.597095 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-70f10453-0c77-4c18-984a-d780d029d010 admin admin] The token cache is being invalidate because project b07da26c1e0a4bdb9b48a96752d67552 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:12:14.597317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-70f10453-0c77-4c18-984a-d780d029d010 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:14.600948 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-70f10453-0c77-4c18-984a-d780d029d010 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:14.606100 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 981/1949] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:12:14 2022] DELETE /identity/v3/projects/b07da26c1e0a4bdb9b48a96752d67552 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:14.612356 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa040e83-2aae-45d3-a619-98aac7915fbd None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:14.612996 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa040e83-2aae-45d3-a619-98aac7915fbd None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:14.613505 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa040e83-2aae-45d3-a619-98aac7915fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:14.647029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-fa040e83-2aae-45d3-a619-98aac7915fbd None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:14.648722 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-fa040e83-2aae-45d3-a619-98aac7915fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:14.706854 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:14.756414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 969/1950] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:14.764906 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e0931-f1f6-4942-87ec-c62e52c167c4 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:14.765232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e0931-f1f6-4942-87ec-c62e52c167c4 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:14.765765 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e0931-f1f6-4942-87ec-c62e52c167c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:14.773016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:14.786369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-257e0931-f1f6-4942-87ec-c62e52c167c4 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:14.787621 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-257e0931-f1f6-4942-87ec-c62e52c167c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:14.800632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 982/1951] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:14.801926 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa040e83-2aae-45d3-a619-98aac7915fbd None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:14.815588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa040e83-2aae-45d3-a619-98aac7915fbd None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:14.821365 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77dc7b-8555-4471-8c36-a197c4e098e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:14.822103 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77dc7b-8555-4471-8c36-a197c4e098e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:14.822395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77dc7b-8555-4471-8c36-a197c4e098e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:14.822835 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77dc7b-8555-4471-8c36-a197c4e098e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:14.823993 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77dc7b-8555-4471-8c36-a197c4e098e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:12:14.824668 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77dc7b-8555-4471-8c36-a197c4e098e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:14.826781 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77dc7b-8555-4471-8c36-a197c4e098e0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:14.841070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 970/1952] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:12:14.845842 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e0931-f1f6-4942-87ec-c62e52c167c4 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:14.856374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e0931-f1f6-4942-87ec-c62e52c167c4 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:14.871802 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77dc7b-8555-4471-8c36-a197c4e098e0 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:14.877073 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6c77dc7b-8555-4471-8c36-a197c4e098e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:14.880347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3c0f4-28e5-480f-a2a1-fe5a5d8c24b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:14.881428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3c0f4-28e5-480f-a2a1-fe5a5d8c24b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:14.883426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3c0f4-28e5-480f-a2a1-fe5a5d8c24b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:14.883426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3c0f4-28e5-480f-a2a1-fe5a5d8c24b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:14.885165 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a3c0f4-28e5-480f-a2a1-fe5a5d8c24b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:14.889023 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a3c0f4-28e5-480f-a2a1-fe5a5d8c24b2 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:14.933456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77dc7b-8555-4471-8c36-a197c4e098e0 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:14.955620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674325fd-4fa5-4648-b324-300f970badcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:14.957235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674325fd-4fa5-4648-b324-300f970badcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:14.958038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674325fd-4fa5-4648-b324-300f970badcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:14.959062 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674325fd-4fa5-4648-b324-300f970badcf admin admin] PATH_INFO: `/v3/projects/f70ea05421084253871d7968389c9533` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:14.960832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674325fd-4fa5-4648-b324-300f970badcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f70ea05421084253871d7968389c9533)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:14.963018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674325fd-4fa5-4648-b324-300f970badcf admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:15.130024 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 983/1953] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:12:14 2022] POST /identity/v3/projects => generated 377 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:15.140285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3c0f4-28e5-480f-a2a1-fe5a5d8c24b2 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:15.193457 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-674325fd-4fa5-4648-b324-300f970badcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f70ea05421084253871d7968389c9533', 'request_id': 'req-674325fd-4fa5-4648-b324-300f970badcf'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:15.214005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-674325fd-4fa5-4648-b324-300f970badcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f70ea05421084253871d7968389c9533', 'request_id': 'req-674325fd-4fa5-4648-b324-300f970badcf'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:15.229581 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-674325fd-4fa5-4648-b324-300f970badcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f70ea05421084253871d7968389c9533', 'request_id': 'req-674325fd-4fa5-4648-b324-300f970badcf'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:15.284895 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3c0f4-28e5-480f-a2a1-fe5a5d8c24b2 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:15.355520 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c03a94-f36c-4b64-9063-75adcfb81bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:15.356271 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c03a94-f36c-4b64-9063-75adcfb81bbd admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:15.356551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c03a94-f36c-4b64-9063-75adcfb81bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:15.356825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c03a94-f36c-4b64-9063-75adcfb81bbd admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:15.357535 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c03a94-f36c-4b64-9063-75adcfb81bbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:15.360070 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c03a94-f36c-4b64-9063-75adcfb81bbd admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:15.379807 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-674325fd-4fa5-4648-b324-300f970badcf admin admin] The token cache is being invalidate because project f70ea05421084253871d7968389c9533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:12:15.380103 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-674325fd-4fa5-4648-b324-300f970badcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:15.380802 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-674325fd-4fa5-4648-b324-300f970badcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:15.388314 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 971/1954] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:12:14 2022] DELETE /identity/v3/projects/f70ea05421084253871d7968389c9533 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:15.388314 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674325fd-4fa5-4648-b324-300f970badcf admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:15.388821 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-674325fd-4fa5-4648-b324-300f970badcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:15.412101 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674325fd-4fa5-4648-b324-300f970badcf admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:15.419808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171a9b3-c89e-4fa3-9da1-e7e00c0e3e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:15.420777 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171a9b3-c89e-4fa3-9da1-e7e00c0e3e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:15.434218 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171a9b3-c89e-4fa3-9da1-e7e00c0e3e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:15.434218 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171a9b3-c89e-4fa3-9da1-e7e00c0e3e50 admin admin] PATH_INFO: `/v3/projects/23652ac7beb34b3a81a710205fe51102` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:15.434218 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171a9b3-c89e-4fa3-9da1-e7e00c0e3e50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23652ac7beb34b3a81a710205fe51102)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:15.434218 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171a9b3-c89e-4fa3-9da1-e7e00c0e3e50 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:15.536463 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 984/1955] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:12:15 2022] POST /identity/v3/users => generated 289 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:15.538619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712e8f6-f4d7-4fa6-b541-c8a12e4479ad None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:15.539525 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712e8f6-f4d7-4fa6-b541-c8a12e4479ad None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:15.539935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712e8f6-f4d7-4fa6-b541-c8a12e4479ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:15.589948 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-2712e8f6-f4d7-4fa6-b541-c8a12e4479ad None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:15.592634 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-2712e8f6-f4d7-4fa6-b541-c8a12e4479ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:15.593580 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b171a9b3-c89e-4fa3-9da1-e7e00c0e3e50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23652ac7beb34b3a81a710205fe51102', 'request_id': 'req-b171a9b3-c89e-4fa3-9da1-e7e00c0e3e50'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:15.621108 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b171a9b3-c89e-4fa3-9da1-e7e00c0e3e50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23652ac7beb34b3a81a710205fe51102', 'request_id': 'req-b171a9b3-c89e-4fa3-9da1-e7e00c0e3e50'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:15.640185 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b171a9b3-c89e-4fa3-9da1-e7e00c0e3e50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23652ac7beb34b3a81a710205fe51102', 'request_id': 'req-b171a9b3-c89e-4fa3-9da1-e7e00c0e3e50'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:15.745755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 985/1956] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:15.747480 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712e8f6-f4d7-4fa6-b541-c8a12e4479ad None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:15.748906 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b171a9b3-c89e-4fa3-9da1-e7e00c0e3e50 admin admin] The token cache is being invalidate because project 23652ac7beb34b3a81a710205fe51102 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:12:15.749651 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b171a9b3-c89e-4fa3-9da1-e7e00c0e3e50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:15.750642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-2712e8f6-f4d7-4fa6-b541-c8a12e4479ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:15.752425 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b171a9b3-c89e-4fa3-9da1-e7e00c0e3e50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:15.760929 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 972/1957] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:12:15 2022] DELETE /identity/v3/projects/23652ac7beb34b3a81a710205fe51102 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:15.763772 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d958d-b697-4a89-aa12-c97a0a83c929 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:15.764912 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d958d-b697-4a89-aa12-c97a0a83c929 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:15.765036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d958d-b697-4a89-aa12-c97a0a83c929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:15.775550 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712e8f6-f4d7-4fa6-b541-c8a12e4479ad None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:15.784840 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0243ca1-b7be-45b6-a200-982a3fc32ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:15.785536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0243ca1-b7be-45b6-a200-982a3fc32ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:15.786032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0243ca1-b7be-45b6-a200-982a3fc32ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:15.786312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0243ca1-b7be-45b6-a200-982a3fc32ad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:15.787100 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0243ca1-b7be-45b6-a200-982a3fc32ad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:15.789123 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0243ca1-b7be-45b6-a200-982a3fc32ad4 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:15.791608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-e82d958d-b697-4a89-aa12-c97a0a83c929 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:15.793272 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e82d958d-b697-4a89-aa12-c97a0a83c929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:15.802924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 986/1958] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:15.804466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 973/1959] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:15.805428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0243ca1-b7be-45b6-a200-982a3fc32ad4 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:15.807860 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d958d-b697-4a89-aa12-c97a0a83c929 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:15.808082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c0243ca1-b7be-45b6-a200-982a3fc32ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:15.809503 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e82d958d-b697-4a89-aa12-c97a0a83c929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:15.824991 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0243ca1-b7be-45b6-a200-982a3fc32ad4 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:15.825270 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d958d-b697-4a89-aa12-c97a0a83c929 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:15.830559 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79136d31-8604-486a-b703-bd6d70bdd2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:15.835515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79136d31-8604-486a-b703-bd6d70bdd2c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:15.835515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79136d31-8604-486a-b703-bd6d70bdd2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:15.835515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79136d31-8604-486a-b703-bd6d70bdd2c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:15.835515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79136d31-8604-486a-b703-bd6d70bdd2c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:15.836071 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4ae868-6b16-45bb-a205-39215938caff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:15.836251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79136d31-8604-486a-b703-bd6d70bdd2c8 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:15.836382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ae868-6b16-45bb-a205-39215938caff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:15.836584 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ae868-6b16-45bb-a205-39215938caff admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:15.837060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4ae868-6b16-45bb-a205-39215938caff admin admin] PATH_INFO: `/v3/users/98576fede6284eb796a4b30d400eea9f` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:15.838658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4ae868-6b16-45bb-a205-39215938caff admin admin] RBAC: Authorizing `identity:delete_user(user_id=98576fede6284eb796a4b30d400eea9f)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:15.841478 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4ae868-6b16-45bb-a205-39215938caff admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:15.855828 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 974/1960] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:15.857355 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79136d31-8604-486a-b703-bd6d70bdd2c8 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:15.875258 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79136d31-8604-486a-b703-bd6d70bdd2c8 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:15.882290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8dd16-ecd0-4cc9-920b-4692a5429a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:15.885159 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8dd16-ecd0-4cc9-920b-4692a5429a77 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:15.885159 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8dd16-ecd0-4cc9-920b-4692a5429a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:15.885159 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8dd16-ecd0-4cc9-920b-4692a5429a77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:15.885159 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8dd16-ecd0-4cc9-920b-4692a5429a77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:12:15.885159 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8dd16-ecd0-4cc9-920b-4692a5429a77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:15.886455 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8dd16-ecd0-4cc9-920b-4692a5429a77 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:15.902319 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 975/1961] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:12:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:12:15.902974 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8dd16-ecd0-4cc9-920b-4692a5429a77 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:15.919293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8dd16-ecd0-4cc9-920b-4692a5429a77 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:15.925788 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8fbe0-cff4-467a-b8f5-e425b2305a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:15.926723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8fbe0-cff4-467a-b8f5-e425b2305a16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:15.927053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8fbe0-cff4-467a-b8f5-e425b2305a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:15.927332 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8fbe0-cff4-467a-b8f5-e425b2305a16 admin admin] PATH_INFO: `/v3/projects/24aacc1c4e2943e39ab19ac312445d44/users/eb220b76c35b46ccb4bdaa617c574880/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:15.946114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8fbe0-cff4-467a-b8f5-e425b2305a16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24aacc1c4e2943e39ab19ac312445d44, user_id=eb220b76c35b46ccb4bdaa617c574880, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:15.949043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8fbe0-cff4-467a-b8f5-e425b2305a16 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:16.035553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:16.057000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-aa4ae868-6b16-45bb-a205-39215938caff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98576fede6284eb796a4b30d400eea9f'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:16.060603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 976/1962] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:15 2022] PUT /identity/v3/projects/24aacc1c4e2943e39ab19ac312445d44/users/eb220b76c35b46ccb4bdaa617c574880/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:16.076311 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8fbe0-cff4-467a-b8f5-e425b2305a16 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:16.118456 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:16.155829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-aa4ae868-6b16-45bb-a205-39215938caff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98576fede6284eb796a4b30d400eea9f'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:16.178068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-aa4ae868-6b16-45bb-a205-39215938caff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98576fede6284eb796a4b30d400eea9f'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:16.201494 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-aa4ae868-6b16-45bb-a205-39215938caff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98576fede6284eb796a4b30d400eea9f'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:16.202677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-aa4ae868-6b16-45bb-a205-39215938caff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98576fede6284eb796a4b30d400eea9f'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:16.204368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8fbe0-cff4-467a-b8f5-e425b2305a16 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:16.212250 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472073ac-17c1-4fbf-a658-e2edc1428fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:16.213321 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472073ac-17c1-4fbf-a658-e2edc1428fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:16.213667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472073ac-17c1-4fbf-a658-e2edc1428fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:16.214041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472073ac-17c1-4fbf-a658-e2edc1428fc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:16.215242 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 987/1963] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:12:15 2022] DELETE /identity/v3/users/98576fede6284eb796a4b30d400eea9f => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:16.215423 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472073ac-17c1-4fbf-a658-e2edc1428fc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:16.218850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472073ac-17c1-4fbf-a658-e2edc1428fc9 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:16.234285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51473f75-92a4-4327-91b5-abc0f37f5819 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:16.235340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51473f75-92a4-4327-91b5-abc0f37f5819 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:16.235930 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51473f75-92a4-4327-91b5-abc0f37f5819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:16.284772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-51473f75-92a4-4327-91b5-abc0f37f5819 None None] MFA Rules not processed for user `eb220b76c35b46ccb4bdaa617c574880`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:16.287478 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-51473f75-92a4-4327-91b5-abc0f37f5819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:16.346303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 977/1964] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:12:16 2022] POST /identity/v3/projects => generated 391 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:16.347867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472073ac-17c1-4fbf-a658-e2edc1428fc9 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:16.350819 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-472073ac-17c1-4fbf-a658-e2edc1428fc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:16.465328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 988/1965] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:16.468088 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51473f75-92a4-4327-91b5-abc0f37f5819 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:16.475911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-51473f75-92a4-4327-91b5-abc0f37f5819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:16.506369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472073ac-17c1-4fbf-a658-e2edc1428fc9 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:16.516551 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17525db3-639a-40cf-9e45-e0ceb424772f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:16.519547 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17525db3-639a-40cf-9e45-e0ceb424772f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:16.520090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17525db3-639a-40cf-9e45-e0ceb424772f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:16.521927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17525db3-639a-40cf-9e45-e0ceb424772f admin admin] PATH_INFO: `/v3/users/38398c84846b4d52a43365a1688db174` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:16.527037 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17525db3-639a-40cf-9e45-e0ceb424772f admin admin] RBAC: Authorizing `identity:delete_user(user_id=38398c84846b4d52a43365a1688db174)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:16.532312 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17525db3-639a-40cf-9e45-e0ceb424772f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:16.635821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51473f75-92a4-4327-91b5-abc0f37f5819 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:16.643323 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530420f8-695e-4412-a51e-5e1b7034eb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:16.645886 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530420f8-695e-4412-a51e-5e1b7034eb9c admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:16.647134 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530420f8-695e-4412-a51e-5e1b7034eb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:16.647810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530420f8-695e-4412-a51e-5e1b7034eb9c admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:16.649282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530420f8-695e-4412-a51e-5e1b7034eb9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:16.653756 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530420f8-695e-4412-a51e-5e1b7034eb9c admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:16.791999 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:16.793071 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 989/1966] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:12:16 2022] POST /identity/v3/users => generated 296 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:16.795901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530420f8-695e-4412-a51e-5e1b7034eb9c admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:16.807198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-530420f8-695e-4412-a51e-5e1b7034eb9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:16.826580 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-17525db3-639a-40cf-9e45-e0ceb424772f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38398c84846b4d52a43365a1688db174'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:16.863886 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530420f8-695e-4412-a51e-5e1b7034eb9c admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:16.885831 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476d5f0-d817-4f6a-888e-e9d731891b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:16.886536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476d5f0-d817-4f6a-888e-e9d731891b09 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:16.886858 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476d5f0-d817-4f6a-888e-e9d731891b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:16.887209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476d5f0-d817-4f6a-888e-e9d731891b09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:16.887929 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476d5f0-d817-4f6a-888e-e9d731891b09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:16.890064 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476d5f0-d817-4f6a-888e-e9d731891b09 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:16.927187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-17525db3-639a-40cf-9e45-e0ceb424772f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38398c84846b4d52a43365a1688db174'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:16.928479 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-17525db3-639a-40cf-9e45-e0ceb424772f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38398c84846b4d52a43365a1688db174'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:16.929105 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-17525db3-639a-40cf-9e45-e0ceb424772f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38398c84846b4d52a43365a1688db174'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:16.930661 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:16.943316 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-17525db3-639a-40cf-9e45-e0ceb424772f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38398c84846b4d52a43365a1688db174'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:16.960011 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 978/1967] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:12:16 2022] DELETE /identity/v3/users/38398c84846b4d52a43365a1688db174 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:16.961196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17525db3-639a-40cf-9e45-e0ceb424772f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:16.964473 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-17525db3-639a-40cf-9e45-e0ceb424772f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:17.050179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 990/1968] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:12:16 2022] POST /identity/v3/projects => generated 377 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:17.050179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8afd5-1cdc-4504-8270-e786321003a3 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:17.050179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8afd5-1cdc-4504-8270-e786321003a3 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:17.050179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8afd5-1cdc-4504-8270-e786321003a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:17.061507 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17525db3-639a-40cf-9e45-e0ceb424772f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:17.072769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-7cb8afd5-1cdc-4504-8270-e786321003a3 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:17.074042 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7cb8afd5-1cdc-4504-8270-e786321003a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:17.149418 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c89cc8-5321-4cea-a235-ae05066ae41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:17.150239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c89cc8-5321-4cea-a235-ae05066ae41c admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:17.150627 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c89cc8-5321-4cea-a235-ae05066ae41c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:17.151016 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c89cc8-5321-4cea-a235-ae05066ae41c admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:17.151944 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c89cc8-5321-4cea-a235-ae05066ae41c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:17.157100 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 991/1969] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:17.157363 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c89cc8-5321-4cea-a235-ae05066ae41c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:17.157542 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8afd5-1cdc-4504-8270-e786321003a3 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:17.157542 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7cb8afd5-1cdc-4504-8270-e786321003a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:17.169931 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 979/1970] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:17.169931 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c89cc8-5321-4cea-a235-ae05066ae41c admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:17.172064 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a1c89cc8-5321-4cea-a235-ae05066ae41c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:17.176443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8afd5-1cdc-4504-8270-e786321003a3 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:17.181717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15abfbe-3fce-4892-a7be-d37bc8622f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:17.182609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15abfbe-3fce-4892-a7be-d37bc8622f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:17.182909 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15abfbe-3fce-4892-a7be-d37bc8622f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:17.183185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15abfbe-3fce-4892-a7be-d37bc8622f9e admin admin] PATH_INFO: `/v3/projects/c63a85df05644010a80d7c5876604b42` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:17.184597 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15abfbe-3fce-4892-a7be-d37bc8622f9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c63a85df05644010a80d7c5876604b42)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:17.186571 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15abfbe-3fce-4892-a7be-d37bc8622f9e admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:17.188290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c89cc8-5321-4cea-a235-ae05066ae41c admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:17.196596 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793031e-d50b-4d72-801b-84787b524637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:17.197301 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793031e-d50b-4d72-801b-84787b524637 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:17.197590 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793031e-d50b-4d72-801b-84787b524637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:17.197867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793031e-d50b-4d72-801b-84787b524637 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:17.198587 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793031e-d50b-4d72-801b-84787b524637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:17.201994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793031e-d50b-4d72-801b-84787b524637 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:17.360011 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 980/1971] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:12:17 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:17.361686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793031e-d50b-4d72-801b-84787b524637 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:17.372437 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c15abfbe-3fce-4892-a7be-d37bc8622f9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c63a85df05644010a80d7c5876604b42', 'request_id': 'req-c15abfbe-3fce-4892-a7be-d37bc8622f9e'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:17.379927 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c15abfbe-3fce-4892-a7be-d37bc8622f9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c63a85df05644010a80d7c5876604b42', 'request_id': 'req-c15abfbe-3fce-4892-a7be-d37bc8622f9e'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:17.380927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793031e-d50b-4d72-801b-84787b524637 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:17.387865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c15abfbe-3fce-4892-a7be-d37bc8622f9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c63a85df05644010a80d7c5876604b42', 'request_id': 'req-c15abfbe-3fce-4892-a7be-d37bc8622f9e'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:17.457627 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a252e42-5140-4b1e-bea6-05e8cd034be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:17.458383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a252e42-5140-4b1e-bea6-05e8cd034be2 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:17.458701 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a252e42-5140-4b1e-bea6-05e8cd034be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:17.459088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a252e42-5140-4b1e-bea6-05e8cd034be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:17.460197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a252e42-5140-4b1e-bea6-05e8cd034be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:17.462207 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a252e42-5140-4b1e-bea6-05e8cd034be2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:17.487006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 981/1972] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:17.488796 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a252e42-5140-4b1e-bea6-05e8cd034be2 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:17.498333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c15abfbe-3fce-4892-a7be-d37bc8622f9e admin admin] The token cache is being invalidate because project c63a85df05644010a80d7c5876604b42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:12:17.498782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c15abfbe-3fce-4892-a7be-d37bc8622f9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:17.500990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c15abfbe-3fce-4892-a7be-d37bc8622f9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:17.503871 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-0a252e42-5140-4b1e-bea6-05e8cd034be2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:17.504083 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 992/1973] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:12:17 2022] DELETE /identity/v3/projects/c63a85df05644010a80d7c5876604b42 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:17.511005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15abfbe-3fce-4892-a7be-d37bc8622f9e admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:17.513137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c15abfbe-3fce-4892-a7be-d37bc8622f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:17.540188 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15abfbe-3fce-4892-a7be-d37bc8622f9e admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:17.546407 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd7433-48e3-4de9-a606-4a3ff693f2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:17.547157 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd7433-48e3-4de9-a606-4a3ff693f2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:17.547442 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd7433-48e3-4de9-a606-4a3ff693f2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:17.547725 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd7433-48e3-4de9-a606-4a3ff693f2b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:17.548480 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd7433-48e3-4de9-a606-4a3ff693f2b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:17.550139 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd7433-48e3-4de9-a606-4a3ff693f2b8 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:17.568570 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 993/1974] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:17.568570 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd7433-48e3-4de9-a606-4a3ff693f2b8 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:17.571440 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-eacd7433-48e3-4de9-a606-4a3ff693f2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:17.613447 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd7433-48e3-4de9-a606-4a3ff693f2b8 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:17.630223 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4d7ba-8af9-401b-a627-4953aa5cedf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:17.631181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4d7ba-8af9-401b-a627-4953aa5cedf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:17.631524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4d7ba-8af9-401b-a627-4953aa5cedf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:17.631803 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4d7ba-8af9-401b-a627-4953aa5cedf9 admin admin] PATH_INFO: `/v3/projects/809727ae4d354ff08d7f20f8f0c133d3/users/116c4f95c0a64ad190f32c64500f7e78/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:17.639055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a252e42-5140-4b1e-bea6-05e8cd034be2 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:17.646302 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9a4fb-66f3-40a4-abfc-3521efc8d8f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:17.647421 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9a4fb-66f3-40a4-abfc-3521efc8d8f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:17.647912 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9a4fb-66f3-40a4-abfc-3521efc8d8f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:17.648401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9a4fb-66f3-40a4-abfc-3521efc8d8f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:17.651077 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf4d7ba-8af9-401b-a627-4953aa5cedf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=809727ae4d354ff08d7f20f8f0c133d3, user_id=116c4f95c0a64ad190f32c64500f7e78, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:17.651895 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-0ec9a4fb-66f3-40a4-abfc-3521efc8d8f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:17.653949 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf4d7ba-8af9-401b-a627-4953aa5cedf9 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:17.672223 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:17.677092 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec9a4fb-66f3-40a4-abfc-3521efc8d8f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:17.679982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec9a4fb-66f3-40a4-abfc-3521efc8d8f6 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:17.699126 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 982/1975] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:17.701573 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bf620-a7e9-4976-ad9e-a4f2bd8e83bc None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:17.702290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bf620-a7e9-4976-ad9e-a4f2bd8e83bc None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:17.703586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bf620-a7e9-4976-ad9e-a4f2bd8e83bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:17.730009 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b87bf620-a7e9-4976-ad9e-a4f2bd8e83bc None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:17.731585 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b87bf620-a7e9-4976-ad9e-a4f2bd8e83bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:17.742642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 983/1976] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:17.743772 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87bf620-a7e9-4976-ad9e-a4f2bd8e83bc None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:17.755854 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87bf620-a7e9-4976-ad9e-a4f2bd8e83bc None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:17.761719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89627d6-6e6f-4677-99d1-bd8e9e978543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:17.762380 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89627d6-6e6f-4677-99d1-bd8e9e978543 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:17.762744 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89627d6-6e6f-4677-99d1-bd8e9e978543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:17.763102 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89627d6-6e6f-4677-99d1-bd8e9e978543 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:17.764595 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89627d6-6e6f-4677-99d1-bd8e9e978543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:17.766999 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89627d6-6e6f-4677-99d1-bd8e9e978543 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:17.781855 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 984/1977] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:17.781855 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fab2bd-ee4e-4a43-b028-a0a183c1d26d None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:17.781855 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fab2bd-ee4e-4a43-b028-a0a183c1d26d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:17.781855 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fab2bd-ee4e-4a43-b028-a0a183c1d26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:17.801219 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-68fab2bd-ee4e-4a43-b028-a0a183c1d26d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:17.802750 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-68fab2bd-ee4e-4a43-b028-a0a183c1d26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:17.802925 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 994/1978] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:17 2022] PUT /identity/v3/projects/809727ae4d354ff08d7f20f8f0c133d3/users/116c4f95c0a64ad190f32c64500f7e78/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:17.803851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4d7ba-8af9-401b-a627-4953aa5cedf9 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:17.835661 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:17.939444 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4d7ba-8af9-401b-a627-4953aa5cedf9 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:17.944273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 985/1979] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:17.944273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec8e8c-213b-416d-9982-628cf4c22215 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:17.944595 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec8e8c-213b-416d-9982-628cf4c22215 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:17.950975 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec8e8c-213b-416d-9982-628cf4c22215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:17.980139 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50973763-496b-4367-9e7c-42a9291cc049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:17.981998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50973763-496b-4367-9e7c-42a9291cc049 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:17.983763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50973763-496b-4367-9e7c-42a9291cc049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:17.988750 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50973763-496b-4367-9e7c-42a9291cc049 admin admin] PATH_INFO: `/v3/projects/12c555244cfc4599a422b00782cebb37/users/f98d98ec55514c14839e4761509d0538/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:18.063170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50973763-496b-4367-9e7c-42a9291cc049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12c555244cfc4599a422b00782cebb37, user_id=f98d98ec55514c14839e4761509d0538, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:18.080463 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50973763-496b-4367-9e7c-42a9291cc049 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:18.081729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-34ec8e8c-213b-416d-9982-628cf4c22215 None None] MFA Rules not processed for user `116c4f95c0a64ad190f32c64500f7e78`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:18.090724 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-34ec8e8c-213b-416d-9982-628cf4c22215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:18.209106 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 995/1980] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:17 2022] PUT /identity/v3/projects/12c555244cfc4599a422b00782cebb37/users/f98d98ec55514c14839e4761509d0538/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:18.209444 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50973763-496b-4367-9e7c-42a9291cc049 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:18.245211 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 986/1981] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:18.247217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3d47a-528f-4453-ae28-e5dd702ba50c None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:18.247836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3d47a-528f-4453-ae28-e5dd702ba50c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:18.248443 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3d47a-528f-4453-ae28-e5dd702ba50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:18.279695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:18.292035 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-14f3d47a-528f-4453-ae28-e5dd702ba50c None None] MFA Rules not processed for user `f98d98ec55514c14839e4761509d0538`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:18.293456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-14f3d47a-528f-4453-ae28-e5dd702ba50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:18.343402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50973763-496b-4367-9e7c-42a9291cc049 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:18.349128 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeded0e9-bcf2-4863-8744-30879fdbe4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:18.350004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeded0e9-bcf2-4863-8744-30879fdbe4eb admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:18.350280 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeded0e9-bcf2-4863-8744-30879fdbe4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:18.350553 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeded0e9-bcf2-4863-8744-30879fdbe4eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:18.351363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeded0e9-bcf2-4863-8744-30879fdbe4eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:12:18.351676 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeded0e9-bcf2-4863-8744-30879fdbe4eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:18.353474 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeded0e9-bcf2-4863-8744-30879fdbe4eb admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:18.369537 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 996/1982] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:12:18.370541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeded0e9-bcf2-4863-8744-30879fdbe4eb admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:18.377601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-aeded0e9-bcf2-4863-8744-30879fdbe4eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:18.438616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 987/1983] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:18.440103 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3d47a-528f-4453-ae28-e5dd702ba50c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:18.461005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3d47a-528f-4453-ae28-e5dd702ba50c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:18.468574 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d588d4-d043-4c55-85ec-bc91e30f180a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:18.469696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d588d4-d043-4c55-85ec-bc91e30f180a admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:18.470172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d588d4-d043-4c55-85ec-bc91e30f180a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:18.470688 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d588d4-d043-4c55-85ec-bc91e30f180a admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:18.471668 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d588d4-d043-4c55-85ec-bc91e30f180a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:18.474547 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d588d4-d043-4c55-85ec-bc91e30f180a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:18.474982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeded0e9-bcf2-4863-8744-30879fdbe4eb admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:18.479151 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540936f2-8178-41ba-bb35-73ddcd539034 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:18.479895 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540936f2-8178-41ba-bb35-73ddcd539034 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:18.480089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540936f2-8178-41ba-bb35-73ddcd539034 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:18.480361 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540936f2-8178-41ba-bb35-73ddcd539034 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:18.548768 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:18.573934 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 988/1984] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:12:18 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:18.575332 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d588d4-d043-4c55-85ec-bc91e30f180a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:18.588404 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540936f2-8178-41ba-bb35-73ddcd539034 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:18.590863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540936f2-8178-41ba-bb35-73ddcd539034 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:18.606849 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 997/1985] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:18.608658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540936f2-8178-41ba-bb35-73ddcd539034 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:18.613154 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-540936f2-8178-41ba-bb35-73ddcd539034 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:18.648172 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540936f2-8178-41ba-bb35-73ddcd539034 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:18.779498 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252afb2-bc9c-4b3e-bfc8-98e95d5bc524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:18.783639 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252afb2-bc9c-4b3e-bfc8-98e95d5bc524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:18.784082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252afb2-bc9c-4b3e-bfc8-98e95d5bc524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:18.784390 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252afb2-bc9c-4b3e-bfc8-98e95d5bc524 admin admin] PATH_INFO: `/v3/projects/0e0a9eeca0b1458dbfaffb22c417f131` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:18.785926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252afb2-bc9c-4b3e-bfc8-98e95d5bc524 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e0a9eeca0b1458dbfaffb22c417f131)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:18.786103 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d588d4-d043-4c55-85ec-bc91e30f180a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:18.794758 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252afb2-bc9c-4b3e-bfc8-98e95d5bc524 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:18.796099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d55290-3b3a-4fb5-9dac-bd3338ebc27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:18.796099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d55290-3b3a-4fb5-9dac-bd3338ebc27f admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:18.796384 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d55290-3b3a-4fb5-9dac-bd3338ebc27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:18.796782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d55290-3b3a-4fb5-9dac-bd3338ebc27f admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:18.798156 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d55290-3b3a-4fb5-9dac-bd3338ebc27f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:18.808035 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d55290-3b3a-4fb5-9dac-bd3338ebc27f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:18.954172 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-2252afb2-bc9c-4b3e-bfc8-98e95d5bc524 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e0a9eeca0b1458dbfaffb22c417f131', 'request_id': 'req-2252afb2-bc9c-4b3e-bfc8-98e95d5bc524'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:18.962041 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-2252afb2-bc9c-4b3e-bfc8-98e95d5bc524 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e0a9eeca0b1458dbfaffb22c417f131', 'request_id': 'req-2252afb2-bc9c-4b3e-bfc8-98e95d5bc524'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:18.967992 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 989/1986] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:12:18 2022] POST /identity/v3/projects => generated 375 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:18.969913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d55290-3b3a-4fb5-9dac-bd3338ebc27f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:18.972310 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-2252afb2-bc9c-4b3e-bfc8-98e95d5bc524 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e0a9eeca0b1458dbfaffb22c417f131', 'request_id': 'req-2252afb2-bc9c-4b3e-bfc8-98e95d5bc524'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:19.055534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d55290-3b3a-4fb5-9dac-bd3338ebc27f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:19.060774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bab039-eacb-4d9e-a775-0df81f8bf0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:19.061633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bab039-eacb-4d9e-a775-0df81f8bf0db admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:19.062105 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bab039-eacb-4d9e-a775-0df81f8bf0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:19.063069 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bab039-eacb-4d9e-a775-0df81f8bf0db admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:19.063813 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bab039-eacb-4d9e-a775-0df81f8bf0db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:19.065801 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bab039-eacb-4d9e-a775-0df81f8bf0db admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:19.075479 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:19.095291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-2252afb2-bc9c-4b3e-bfc8-98e95d5bc524 admin admin] The token cache is being invalidate because project 0e0a9eeca0b1458dbfaffb22c417f131 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:12:19.095729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-2252afb2-bc9c-4b3e-bfc8-98e95d5bc524 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:19.097598 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-2252afb2-bc9c-4b3e-bfc8-98e95d5bc524 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:19.100162 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 998/1987] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:12:18 2022] DELETE /identity/v3/projects/0e0a9eeca0b1458dbfaffb22c417f131 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:19.101399 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252afb2-bc9c-4b3e-bfc8-98e95d5bc524 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:19.103656 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-2252afb2-bc9c-4b3e-bfc8-98e95d5bc524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:19.193438 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 990/1988] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:12:18 2022] POST /identity/v3/users => generated 292 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:19.194826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ff3e8-7b3f-4928-b3a8-33c324d2738c None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:19.195132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ff3e8-7b3f-4928-b3a8-33c324d2738c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:19.195407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ff3e8-7b3f-4928-b3a8-33c324d2738c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:19.210442 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252afb2-bc9c-4b3e-bfc8-98e95d5bc524 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:19.219527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f50b7-2f8b-47b1-a34a-bee87c721d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:19.221145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f50b7-2f8b-47b1-a34a-bee87c721d44 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:19.221689 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f50b7-2f8b-47b1-a34a-bee87c721d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:19.222201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f50b7-2f8b-47b1-a34a-bee87c721d44 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:19.223246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f50b7-2f8b-47b1-a34a-bee87c721d44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:19.225528 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f50b7-2f8b-47b1-a34a-bee87c721d44 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:19.243412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-057ff3e8-7b3f-4928-b3a8-33c324d2738c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:19.247558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-057ff3e8-7b3f-4928-b3a8-33c324d2738c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:19.261934 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 991/1989] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:19.263416 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ff3e8-7b3f-4928-b3a8-33c324d2738c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:19.271811 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-057ff3e8-7b3f-4928-b3a8-33c324d2738c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:19.293895 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ff3e8-7b3f-4928-b3a8-33c324d2738c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:19.300491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15327e25-4356-4075-b364-9768173eb7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:19.301258 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15327e25-4356-4075-b364-9768173eb7c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:19.301473 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15327e25-4356-4075-b364-9768173eb7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:19.301717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15327e25-4356-4075-b364-9768173eb7c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:19.302454 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15327e25-4356-4075-b364-9768173eb7c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:19.304276 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15327e25-4356-4075-b364-9768173eb7c0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:19.317014 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 992/1990] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:19.320904 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 999/1991] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:12:19 2022] POST /identity/v3/users => generated 294 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:19.321819 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f97d25-7566-4bf5-a97a-13421999e63f None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:19.321819 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f97d25-7566-4bf5-a97a-13421999e63f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:19.321819 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f97d25-7566-4bf5-a97a-13421999e63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:19.322067 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f50b7-2f8b-47b1-a34a-bee87c721d44 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:19.335261 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f50b7-2f8b-47b1-a34a-bee87c721d44 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:19.340578 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a33ad-9b81-412d-9fd1-5e6024aff900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:19.341370 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a33ad-9b81-412d-9fd1-5e6024aff900 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:19.342384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a33ad-9b81-412d-9fd1-5e6024aff900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:19.342384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a33ad-9b81-412d-9fd1-5e6024aff900 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:19.342728 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a33ad-9b81-412d-9fd1-5e6024aff900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:19.345644 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a33ad-9b81-412d-9fd1-5e6024aff900 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:19.346384 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-11f97d25-7566-4bf5-a97a-13421999e63f None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:19.348030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-11f97d25-7566-4bf5-a97a-13421999e63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:19.348807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:19.361173 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 993/1992] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:19.362486 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f97d25-7566-4bf5-a97a-13421999e63f None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:19.368937 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1000/1993] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:19.368937 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a33ad-9b81-412d-9fd1-5e6024aff900 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:19.378531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f97d25-7566-4bf5-a97a-13421999e63f None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:19.385950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb957597-1794-4699-b880-559e9688de0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:19.387096 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb957597-1794-4699-b880-559e9688de0c admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:19.392427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb957597-1794-4699-b880-559e9688de0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:19.392427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb957597-1794-4699-b880-559e9688de0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:19.392427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb957597-1794-4699-b880-559e9688de0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:19.392859 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a33ad-9b81-412d-9fd1-5e6024aff900 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:19.392943 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb957597-1794-4699-b880-559e9688de0c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:19.404713 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766fda7-3f00-4494-bd08-07b02b999500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:19.405451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766fda7-3f00-4494-bd08-07b02b999500 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:19.405736 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766fda7-3f00-4494-bd08-07b02b999500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:19.406244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766fda7-3f00-4494-bd08-07b02b999500 admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:19.408924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c766fda7-3f00-4494-bd08-07b02b999500 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:12:19.409297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c766fda7-3f00-4494-bd08-07b02b999500 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:19.411063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c766fda7-3f00-4494-bd08-07b02b999500 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:19.418327 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 994/1994] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:19.419179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb957597-1794-4699-b880-559e9688de0c admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:19.429186 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1001/1995] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:12:19.430050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766fda7-3f00-4494-bd08-07b02b999500 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:19.432859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb957597-1794-4699-b880-559e9688de0c admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:19.439570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c29810-4c89-412a-a3e4-7abfee845250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:19.443952 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c29810-4c89-412a-a3e4-7abfee845250 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:19.443952 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c29810-4c89-412a-a3e4-7abfee845250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:19.443952 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c29810-4c89-412a-a3e4-7abfee845250 admin admin] PATH_INFO: `/v3/projects/d566f74e0d83492aad3bfcee2f4c7d67/users/e7b8b164d575415795203ff63a9398cd/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:19.450364 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766fda7-3f00-4494-bd08-07b02b999500 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:19.457549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a517d4-876f-4e9f-be64-c38368d05523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:19.457843 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a517d4-876f-4e9f-be64-c38368d05523 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:19.458200 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a517d4-876f-4e9f-be64-c38368d05523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:19.458381 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a517d4-876f-4e9f-be64-c38368d05523 admin admin] PATH_INFO: `/v3/projects/11a233b00bfb4a10acc46f21156d1baa/users/af117bf9ef5b4e3b87deb2f896cd01cc/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:19.464720 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c29810-4c89-412a-a3e4-7abfee845250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d566f74e0d83492aad3bfcee2f4c7d67, user_id=e7b8b164d575415795203ff63a9398cd, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:19.467832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c29810-4c89-412a-a3e4-7abfee845250 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:19.478795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a517d4-876f-4e9f-be64-c38368d05523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11a233b00bfb4a10acc46f21156d1baa, user_id=af117bf9ef5b4e3b87deb2f896cd01cc, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:19.481646 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a517d4-876f-4e9f-be64-c38368d05523 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:19.600270 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 995/1996] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:19 2022] PUT /identity/v3/projects/d566f74e0d83492aad3bfcee2f4c7d67/users/e7b8b164d575415795203ff63a9398cd/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:19.600779 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c29810-4c89-412a-a3e4-7abfee845250 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:19.638304 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1002/1997] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:19 2022] PUT /identity/v3/projects/11a233b00bfb4a10acc46f21156d1baa/users/af117bf9ef5b4e3b87deb2f896cd01cc/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:19.638304 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59f0ea-efdf-4f8e-9be3-fe330c0ac228 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:19.638676 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59f0ea-efdf-4f8e-9be3-fe330c0ac228 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:19.638987 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59f0ea-efdf-4f8e-9be3-fe330c0ac228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:19.705255 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-0c59f0ea-efdf-4f8e-9be3-fe330c0ac228 None None] MFA Rules not processed for user `e7b8b164d575415795203ff63a9398cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:19.707859 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-0c59f0ea-efdf-4f8e-9be3-fe330c0ac228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:19.713835 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c29810-4c89-412a-a3e4-7abfee845250 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:19.806102 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206d448-24dd-4060-805e-c12c9e7737f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:19.807284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206d448-24dd-4060-805e-c12c9e7737f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:19.807284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206d448-24dd-4060-805e-c12c9e7737f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:19.807460 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206d448-24dd-4060-805e-c12c9e7737f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:19.808715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206d448-24dd-4060-805e-c12c9e7737f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:19.810845 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206d448-24dd-4060-805e-c12c9e7737f0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:19.847189 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1003/1998] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:19.848053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59f0ea-efdf-4f8e-9be3-fe330c0ac228 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:19.859034 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59f0ea-efdf-4f8e-9be3-fe330c0ac228 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:19.864391 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bd8d0-0d92-40d7-bc5f-d9fb664c6505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:19.865154 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bd8d0-0d92-40d7-bc5f-d9fb664c6505 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:19.865443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bd8d0-0d92-40d7-bc5f-d9fb664c6505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:19.865720 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bd8d0-0d92-40d7-bc5f-d9fb664c6505 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:19.866456 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bd8d0-0d92-40d7-bc5f-d9fb664c6505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:19.868139 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bd8d0-0d92-40d7-bc5f-d9fb664c6505 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:19.880505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1004/1999] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:19.881374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bd8d0-0d92-40d7-bc5f-d9fb664c6505 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:19.884405 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-768bd8d0-0d92-40d7-bc5f-d9fb664c6505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:19.898442 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bd8d0-0d92-40d7-bc5f-d9fb664c6505 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:19.903884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39783f5d-523f-4b3c-9a44-10884488578b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:19.904794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39783f5d-523f-4b3c-9a44-10884488578b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:19.905079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39783f5d-523f-4b3c-9a44-10884488578b admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:19.905356 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39783f5d-523f-4b3c-9a44-10884488578b admin admin] PATH_INFO: `/v3/users/19104226c2a64fcab522a5f2ade6fc3a` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:19.907758 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39783f5d-523f-4b3c-9a44-10884488578b admin admin] RBAC: Authorizing `identity:delete_user(user_id=19104226c2a64fcab522a5f2ade6fc3a)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:19.908937 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39783f5d-523f-4b3c-9a44-10884488578b admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:19.956930 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 996/2000] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:12:19 2022] POST /identity/v3/projects => generated 371 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:19.957946 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206d448-24dd-4060-805e-c12c9e7737f0 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:19.988251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:20.034608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206d448-24dd-4060-805e-c12c9e7737f0 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:20.040334 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328cbdb-a1ba-41e7-b8d2-b9c0303ab500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:20.041063 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328cbdb-a1ba-41e7-b8d2-b9c0303ab500 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:20.041444 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328cbdb-a1ba-41e7-b8d2-b9c0303ab500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:20.041900 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328cbdb-a1ba-41e7-b8d2-b9c0303ab500 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:20.042671 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328cbdb-a1ba-41e7-b8d2-b9c0303ab500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:20.044822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9328cbdb-a1ba-41e7-b8d2-b9c0303ab500 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:20.131910 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-39783f5d-523f-4b3c-9a44-10884488578b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19104226c2a64fcab522a5f2ade6fc3a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:20.152578 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 997/2001] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:12:19 2022] POST /identity/v3/projects => generated 381 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:20.153727 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328cbdb-a1ba-41e7-b8d2-b9c0303ab500 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:20.212375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-39783f5d-523f-4b3c-9a44-10884488578b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19104226c2a64fcab522a5f2ade6fc3a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:20.223825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-39783f5d-523f-4b3c-9a44-10884488578b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19104226c2a64fcab522a5f2ade6fc3a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:20.228393 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328cbdb-a1ba-41e7-b8d2-b9c0303ab500 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:20.233966 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005848dc-b4d6-4ddf-9bfd-6f9894c25c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:20.235852 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005848dc-b4d6-4ddf-9bfd-6f9894c25c4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:20.236136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005848dc-b4d6-4ddf-9bfd-6f9894c25c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:20.236412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005848dc-b4d6-4ddf-9bfd-6f9894c25c4f admin admin] PATH_INFO: `/v3/projects/11a233b00bfb4a10acc46f21156d1baa/users/af117bf9ef5b4e3b87deb2f896cd01cc/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:20.238475 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-39783f5d-523f-4b3c-9a44-10884488578b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19104226c2a64fcab522a5f2ade6fc3a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:20.238606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005848dc-b4d6-4ddf-9bfd-6f9894c25c4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=11a233b00bfb4a10acc46f21156d1baa, user_id=af117bf9ef5b4e3b87deb2f896cd01cc, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:20.239874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-39783f5d-523f-4b3c-9a44-10884488578b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19104226c2a64fcab522a5f2ade6fc3a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:20.241237 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005848dc-b4d6-4ddf-9bfd-6f9894c25c4f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:20.246832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1005/2002] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:12:19 2022] DELETE /identity/v3/users/19104226c2a64fcab522a5f2ade6fc3a => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:20.250780 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39783f5d-523f-4b3c-9a44-10884488578b admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:20.250780 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-39783f5d-523f-4b3c-9a44-10884488578b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:20.257699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 998/2003] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:20 2022] PUT /identity/v3/projects/11a233b00bfb4a10acc46f21156d1baa/users/af117bf9ef5b4e3b87deb2f896cd01cc/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:20.258691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005848dc-b4d6-4ddf-9bfd-6f9894c25c4f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:20.269055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-005848dc-b4d6-4ddf-9bfd-6f9894c25c4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:20.277064 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:20.349266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39783f5d-523f-4b3c-9a44-10884488578b admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:20.400337 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005848dc-b4d6-4ddf-9bfd-6f9894c25c4f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:20.411604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeee556-4093-4c0a-8e93-1eb9f5f65930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:20.412818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeee556-4093-4c0a-8e93-1eb9f5f65930 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:20.413382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeee556-4093-4c0a-8e93-1eb9f5f65930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:20.413844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeee556-4093-4c0a-8e93-1eb9f5f65930 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:20.415263 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeee556-4093-4c0a-8e93-1eb9f5f65930 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:20.419428 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeee556-4093-4c0a-8e93-1eb9f5f65930 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:20.425715 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e76f4-56b0-4a60-8ec2-e9be3d8d0c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:20.425846 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e76f4-56b0-4a60-8ec2-e9be3d8d0c4e admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:20.426124 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e76f4-56b0-4a60-8ec2-e9be3d8d0c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:20.426821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e76f4-56b0-4a60-8ec2-e9be3d8d0c4e admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:20.427310 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0e76f4-56b0-4a60-8ec2-e9be3d8d0c4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:20.429308 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0e76f4-56b0-4a60-8ec2-e9be3d8d0c4e admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:20.555436 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 999/2004] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:12:20 2022] POST /identity/v3/users => generated 297 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:20.555436 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803870f-1278-4550-a936-07aa8717a4cf None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:20.555436 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803870f-1278-4550-a936-07aa8717a4cf None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:20.555436 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803870f-1278-4550-a936-07aa8717a4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:20.556270 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1006/2005] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:12:20 2022] POST /identity/v3/users => generated 286 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:20.558939 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e76f4-56b0-4a60-8ec2-e9be3d8d0c4e admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:20.577662 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e76f4-56b0-4a60-8ec2-e9be3d8d0c4e admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:20.586107 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc98617-ac49-404f-a63c-9ff6597bacc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:20.587093 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-6803870f-1278-4550-a936-07aa8717a4cf None None] MFA Rules not processed for user `af117bf9ef5b4e3b87deb2f896cd01cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:20.587224 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc98617-ac49-404f-a63c-9ff6597bacc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:20.587695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc98617-ac49-404f-a63c-9ff6597bacc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:20.588364 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc98617-ac49-404f-a63c-9ff6597bacc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:20.589598 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc98617-ac49-404f-a63c-9ff6597bacc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:20.592915 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc98617-ac49-404f-a63c-9ff6597bacc2 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:20.595882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6803870f-1278-4550-a936-07aa8717a4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:20.635888 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:20.640353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1007/2006] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:12:20.642634 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc98617-ac49-404f-a63c-9ff6597bacc2 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:20.655807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc98617-ac49-404f-a63c-9ff6597bacc2 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:20.661847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de6692-a778-4ade-b239-0fb3c22d7f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:20.664540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de6692-a778-4ade-b239-0fb3c22d7f53 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:20.664540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de6692-a778-4ade-b239-0fb3c22d7f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:20.664540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de6692-a778-4ade-b239-0fb3c22d7f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:20.664540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de6692-a778-4ade-b239-0fb3c22d7f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:20.665610 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de6692-a778-4ade-b239-0fb3c22d7f53 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:20.684700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1008/2007] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:20.684700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de6692-a778-4ade-b239-0fb3c22d7f53 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:20.684700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-44de6692-a778-4ade-b239-0fb3c22d7f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:20.706632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de6692-a778-4ade-b239-0fb3c22d7f53 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:20.716148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525cde60-267b-427d-8beb-d597092f1e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:20.716148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cde60-267b-427d-8beb-d597092f1e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:20.716148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cde60-267b-427d-8beb-d597092f1e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:20.716148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cde60-267b-427d-8beb-d597092f1e98 admin admin] PATH_INFO: `/v3/projects/909b9d1ab80d456b8801692973d6799d` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:20.717576 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525cde60-267b-427d-8beb-d597092f1e98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=909b9d1ab80d456b8801692973d6799d)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:20.720617 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525cde60-267b-427d-8beb-d597092f1e98 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:20.724255 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1000/2008] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:20.724255 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803870f-1278-4550-a936-07aa8717a4cf None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:20.737007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803870f-1278-4550-a936-07aa8717a4cf None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:20.742269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575cf72-f467-4079-8e78-200823841ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:20.743001 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575cf72-f467-4079-8e78-200823841ead admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:20.743285 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575cf72-f467-4079-8e78-200823841ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:20.743554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575cf72-f467-4079-8e78-200823841ead admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:20.744306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a575cf72-f467-4079-8e78-200823841ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:20.746005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a575cf72-f467-4079-8e78-200823841ead admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:20.758177 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1001/2009] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:20.762087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575cf72-f467-4079-8e78-200823841ead admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:20.770792 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575cf72-f467-4079-8e78-200823841ead admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:20.779350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ede25-c4b4-4726-ba4a-36ee03864043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:20.780291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ede25-c4b4-4726-ba4a-36ee03864043 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:20.780572 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ede25-c4b4-4726-ba4a-36ee03864043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:20.780845 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ede25-c4b4-4726-ba4a-36ee03864043 admin admin] PATH_INFO: `/v3/projects/795f7adf639d491f869644b7d5fb2748/users/8e07996ae3b04ff9a891387e1940dff0/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:20.806201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ede25-c4b4-4726-ba4a-36ee03864043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=795f7adf639d491f869644b7d5fb2748, user_id=8e07996ae3b04ff9a891387e1940dff0, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:20.809165 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ede25-c4b4-4726-ba4a-36ee03864043 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:20.908409 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1002/2010] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:20 2022] PUT /identity/v3/projects/795f7adf639d491f869644b7d5fb2748/users/8e07996ae3b04ff9a891387e1940dff0/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:20.909575 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ede25-c4b4-4726-ba4a-36ee03864043 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:20.911839 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-fd2ede25-c4b4-4726-ba4a-36ee03864043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:20.918204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-525cde60-267b-427d-8beb-d597092f1e98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '909b9d1ab80d456b8801692973d6799d', 'request_id': 'req-525cde60-267b-427d-8beb-d597092f1e98'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:20.944706 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-525cde60-267b-427d-8beb-d597092f1e98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '909b9d1ab80d456b8801692973d6799d', 'request_id': 'req-525cde60-267b-427d-8beb-d597092f1e98'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:20.956056 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-525cde60-267b-427d-8beb-d597092f1e98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '909b9d1ab80d456b8801692973d6799d', 'request_id': 'req-525cde60-267b-427d-8beb-d597092f1e98'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:20.991426 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ede25-c4b4-4726-ba4a-36ee03864043 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:20.995432 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc1c7f-8f20-4393-93c4-aec57cd75532 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:20.996090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc1c7f-8f20-4393-93c4-aec57cd75532 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:20.996375 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc1c7f-8f20-4393-93c4-aec57cd75532 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:20.996651 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc1c7f-8f20-4393-93c4-aec57cd75532 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:20.999248 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b0cc1c7f-8f20-4393-93c4-aec57cd75532 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:21.060459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-525cde60-267b-427d-8beb-d597092f1e98 admin admin] The token cache is being invalidate because project 909b9d1ab80d456b8801692973d6799d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:12:21.061062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-525cde60-267b-427d-8beb-d597092f1e98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:21.061745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-525cde60-267b-427d-8beb-d597092f1e98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:21.063815 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1009/2011] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:12:20 2022] DELETE /identity/v3/projects/909b9d1ab80d456b8801692973d6799d => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:21.064731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525cde60-267b-427d-8beb-d597092f1e98 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:21.066556 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-525cde60-267b-427d-8beb-d597092f1e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:21.079014 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc1c7f-8f20-4393-93c4-aec57cd75532 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:21.079238 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:21.080478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc1c7f-8f20-4393-93c4-aec57cd75532 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:21.091387 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1003/2012] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:21.092177 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc1c7f-8f20-4393-93c4-aec57cd75532 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:21.142900 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:21.169458 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525cde60-267b-427d-8beb-d597092f1e98 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:21.173481 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc1c7f-8f20-4393-93c4-aec57cd75532 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:21.177244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96814df-d307-4b2f-9c3e-806337bba1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:21.178927 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96814df-d307-4b2f-9c3e-806337bba1af admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:21.179218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96814df-d307-4b2f-9c3e-806337bba1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:21.179309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14807c-6c57-4f0e-a573-afb0e616a64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:21.179502 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96814df-d307-4b2f-9c3e-806337bba1af admin admin] PATH_INFO: `/v3/projects/00e03c837cb7482483081599eed535b1/users/8ba8498f6fbc40c18cadbb4ce6667855/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:21.180515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14807c-6c57-4f0e-a573-afb0e616a64e admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:21.181159 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14807c-6c57-4f0e-a573-afb0e616a64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:21.182022 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14807c-6c57-4f0e-a573-afb0e616a64e admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:21.182808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14807c-6c57-4f0e-a573-afb0e616a64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:21.185255 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14807c-6c57-4f0e-a573-afb0e616a64e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:21.204260 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1004/2013] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:21.204260 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd769d2-fc08-420e-a077-46b9c53464d8 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:21.204260 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd769d2-fc08-420e-a077-46b9c53464d8 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:21.204260 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd769d2-fc08-420e-a077-46b9c53464d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:21.207594 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96814df-d307-4b2f-9c3e-806337bba1af admin admin] RBAC: Authorizing `identity:create_grant(project_id=00e03c837cb7482483081599eed535b1, user_id=8ba8498f6fbc40c18cadbb4ce6667855, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:21.211826 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96814df-d307-4b2f-9c3e-806337bba1af admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:21.226302 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3fd769d2-fc08-420e-a077-46b9c53464d8 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:21.227771 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3fd769d2-fc08-420e-a077-46b9c53464d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:21.237865 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1005/2014] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:21.238465 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd769d2-fc08-420e-a077-46b9c53464d8 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:21.239901 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3fd769d2-fc08-420e-a077-46b9c53464d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:21.252886 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd769d2-fc08-420e-a077-46b9c53464d8 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:21.258978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139885d3-67a6-4aac-b47c-a162568fd5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:21.260003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139885d3-67a6-4aac-b47c-a162568fd5ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:21.260291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139885d3-67a6-4aac-b47c-a162568fd5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:21.260567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139885d3-67a6-4aac-b47c-a162568fd5ec admin admin] PATH_INFO: `/v3/projects/795f7adf639d491f869644b7d5fb2748/users/8e07996ae3b04ff9a891387e1940dff0/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:21.262733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139885d3-67a6-4aac-b47c-a162568fd5ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=795f7adf639d491f869644b7d5fb2748, user_id=8e07996ae3b04ff9a891387e1940dff0, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:21.265516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139885d3-67a6-4aac-b47c-a162568fd5ec admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:21.278245 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1006/2015] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:21 2022] PUT /identity/v3/projects/795f7adf639d491f869644b7d5fb2748/users/8e07996ae3b04ff9a891387e1940dff0/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:21.279597 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17594f18-6fb1-4a01-84da-4fb509920c5a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:21.284050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17594f18-6fb1-4a01-84da-4fb509920c5a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:21.284050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17594f18-6fb1-4a01-84da-4fb509920c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:21.303528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-17594f18-6fb1-4a01-84da-4fb509920c5a None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:21.304546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-17594f18-6fb1-4a01-84da-4fb509920c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:21.311813 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1010/2016] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:21 2022] PUT /identity/v3/projects/00e03c837cb7482483081599eed535b1/users/8ba8498f6fbc40c18cadbb4ce6667855/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:21.314547 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679fc9d-4e7e-4437-acbe-9ea1416f7194 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:21.314866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679fc9d-4e7e-4437-acbe-9ea1416f7194 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:21.315207 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679fc9d-4e7e-4437-acbe-9ea1416f7194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:21.348261 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-5679fc9d-4e7e-4437-acbe-9ea1416f7194 None None] MFA Rules not processed for user `8e07996ae3b04ff9a891387e1940dff0`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:21.349910 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5679fc9d-4e7e-4437-acbe-9ea1416f7194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:21.416137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1007/2017] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:21.417534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b79cd-299a-438c-8b79-1a2ada4cbcca None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:21.417823 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b79cd-299a-438c-8b79-1a2ada4cbcca None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:21.418092 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b79cd-299a-438c-8b79-1a2ada4cbcca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:21.448312 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-4b9b79cd-299a-438c-8b79-1a2ada4cbcca None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:21.450404 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4b9b79cd-299a-438c-8b79-1a2ada4cbcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:21.463864 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1008/2018] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:21.464158 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1011/2019] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:21.464875 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b79cd-299a-438c-8b79-1a2ada4cbcca None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:21.476303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679fc9d-4e7e-4437-acbe-9ea1416f7194 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:21.477931 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5679fc9d-4e7e-4437-acbe-9ea1416f7194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:21.478373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b79cd-299a-438c-8b79-1a2ada4cbcca None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:21.483404 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee2e69e-9792-46fa-b167-fc0113e89136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:21.484079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2e69e-9792-46fa-b167-fc0113e89136 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:21.485019 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2e69e-9792-46fa-b167-fc0113e89136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:21.485190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2e69e-9792-46fa-b167-fc0113e89136 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:21.485930 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee2e69e-9792-46fa-b167-fc0113e89136 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:12:21.486204 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee2e69e-9792-46fa-b167-fc0113e89136 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:21.487878 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee2e69e-9792-46fa-b167-fc0113e89136 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:21.502033 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1009/2020] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:12:21.502950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee2e69e-9792-46fa-b167-fc0113e89136 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:21.505127 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-fee2e69e-9792-46fa-b167-fc0113e89136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:21.556319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679fc9d-4e7e-4437-acbe-9ea1416f7194 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:21.568068 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948c701-062a-49df-8745-2188347ce11e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:21.568736 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948c701-062a-49df-8745-2188347ce11e service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:21.569012 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948c701-062a-49df-8745-2188347ce11e service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:21.569284 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948c701-062a-49df-8745-2188347ce11e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:21.579959 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee2e69e-9792-46fa-b167-fc0113e89136 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:21.586644 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08478d85-7b04-483f-b4cc-68fa1672108a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:21.587815 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08478d85-7b04-483f-b4cc-68fa1672108a service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:21.588021 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08478d85-7b04-483f-b4cc-68fa1672108a service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:21.588418 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08478d85-7b04-483f-b4cc-68fa1672108a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:21.590123 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948c701-062a-49df-8745-2188347ce11e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:21.591713 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948c701-062a-49df-8745-2188347ce11e service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:21.600161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08478d85-7b04-483f-b4cc-68fa1672108a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:21.601632 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08478d85-7b04-483f-b4cc-68fa1672108a service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:21.603865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1012/2021] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:12:21.604696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948c701-062a-49df-8745-2188347ce11e service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:21.616391 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948c701-062a-49df-8745-2188347ce11e service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:21.617390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1010/2022] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:21.622502 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26918f4-e256-4e43-a03a-3dc2f0ad955d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:21.622502 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26918f4-e256-4e43-a03a-3dc2f0ad955d admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:21.622808 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26918f4-e256-4e43-a03a-3dc2f0ad955d admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:21.622960 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26918f4-e256-4e43-a03a-3dc2f0ad955d admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:21.623685 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26918f4-e256-4e43-a03a-3dc2f0ad955d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:21.627174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26918f4-e256-4e43-a03a-3dc2f0ad955d admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:21.723148 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08478d85-7b04-483f-b4cc-68fa1672108a service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:21.732182 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08478d85-7b04-483f-b4cc-68fa1672108a service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:21.737172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec7d12-98d7-429a-b576-48d7fae7ae02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:21.737822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec7d12-98d7-429a-b576-48d7fae7ae02 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:21.738101 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec7d12-98d7-429a-b576-48d7fae7ae02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:21.738440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec7d12-98d7-429a-b576-48d7fae7ae02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:21.740300 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-99ec7d12-98d7-429a-b576-48d7fae7ae02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:21.757769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:21.768633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1013/2023] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:12:21 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:21.772671 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26918f4-e256-4e43-a03a-3dc2f0ad955d admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:21.794963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:21.860930 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec7d12-98d7-429a-b576-48d7fae7ae02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:21.862806 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec7d12-98d7-429a-b576-48d7fae7ae02 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:21.875607 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1011/2024] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:12:21.899364 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26918f4-e256-4e43-a03a-3dc2f0ad955d admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:21.909759 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3beaa5-15fb-4202-9cd7-39d6b0b2ee45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:21.910512 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3beaa5-15fb-4202-9cd7-39d6b0b2ee45 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:21.910938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3beaa5-15fb-4202-9cd7-39d6b0b2ee45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:21.911229 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3beaa5-15fb-4202-9cd7-39d6b0b2ee45 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:21.911965 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3beaa5-15fb-4202-9cd7-39d6b0b2ee45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:21.914023 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3beaa5-15fb-4202-9cd7-39d6b0b2ee45 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:22.040738 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1014/2025] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:12:21 2022] POST /identity/v3/users => generated 288 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:12:22.044534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec7d12-98d7-429a-b576-48d7fae7ae02 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:22.068481 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec7d12-98d7-429a-b576-48d7fae7ae02 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:22.076940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6e7d7-32c9-4a8c-8119-c1a3eca2a3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:22.079207 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6e7d7-32c9-4a8c-8119-c1a3eca2a3a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:22.079996 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6e7d7-32c9-4a8c-8119-c1a3eca2a3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:22.080502 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6e7d7-32c9-4a8c-8119-c1a3eca2a3a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:22.081817 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6e7d7-32c9-4a8c-8119-c1a3eca2a3a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:22.084558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a6e7d7-32c9-4a8c-8119-c1a3eca2a3a6 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:22.097258 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1012/2026] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:22.105227 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3beaa5-15fb-4202-9cd7-39d6b0b2ee45 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:22.126099 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3beaa5-15fb-4202-9cd7-39d6b0b2ee45 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:22.132412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07252453-1b65-473a-8df2-3c465d3fe34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:22.143752 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07252453-1b65-473a-8df2-3c465d3fe34d admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:22.144148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07252453-1b65-473a-8df2-3c465d3fe34d admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:22.144608 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07252453-1b65-473a-8df2-3c465d3fe34d admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:22.145627 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07252453-1b65-473a-8df2-3c465d3fe34d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:22.147501 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07252453-1b65-473a-8df2-3c465d3fe34d admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:22.161487 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1015/2027] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:22.167650 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6e7d7-32c9-4a8c-8119-c1a3eca2a3a6 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:22.182186 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6e7d7-32c9-4a8c-8119-c1a3eca2a3a6 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:22.188880 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85f2b5-7e19-40c7-8027-1945eeae3aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:22.189806 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85f2b5-7e19-40c7-8027-1945eeae3aa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:22.190087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85f2b5-7e19-40c7-8027-1945eeae3aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:22.190353 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85f2b5-7e19-40c7-8027-1945eeae3aa0 admin admin] PATH_INFO: `/v3/projects/fa53787b85ab4869a4bb67f68b3dde86/users/4cadd4bc4d36437484f495d0e49d9881/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:22.210206 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85f2b5-7e19-40c7-8027-1945eeae3aa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa53787b85ab4869a4bb67f68b3dde86, user_id=4cadd4bc4d36437484f495d0e49d9881, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:22.212939 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85f2b5-7e19-40c7-8027-1945eeae3aa0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:22.352822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1013/2028] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:22 2022] PUT /identity/v3/projects/fa53787b85ab4869a4bb67f68b3dde86/users/4cadd4bc4d36437484f495d0e49d9881/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:12:22.361664 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ab872-bd2e-48af-b78c-32200ac9b857 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:22.362042 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ab872-bd2e-48af-b78c-32200ac9b857 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:22.362383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ab872-bd2e-48af-b78c-32200ac9b857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:22.396881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-486ab872-bd2e-48af-b78c-32200ac9b857 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:22.399404 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-486ab872-bd2e-48af-b78c-32200ac9b857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:22.518138 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1016/2029] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:22.529002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85f2b5-7e19-40c7-8027-1945eeae3aa0 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:22.595770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85f2b5-7e19-40c7-8027-1945eeae3aa0 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:22.600374 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00132e-24bd-46dd-8fbe-bbe0a053891c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:22.601064 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00132e-24bd-46dd-8fbe-bbe0a053891c service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:22.601349 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00132e-24bd-46dd-8fbe-bbe0a053891c service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:22.601627 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00132e-24bd-46dd-8fbe-bbe0a053891c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:22.613548 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00132e-24bd-46dd-8fbe-bbe0a053891c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:22.614922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00132e-24bd-46dd-8fbe-bbe0a053891c service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:22.627742 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1014/2030] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:23.378079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132b8f5-1805-4aff-b509-38a9db8b1206 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:23.378465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132b8f5-1805-4aff-b509-38a9db8b1206 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:23.378640 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132b8f5-1805-4aff-b509-38a9db8b1206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:23.408524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-a132b8f5-1805-4aff-b509-38a9db8b1206 None None] MFA Rules not processed for user `8ba8498f6fbc40c18cadbb4ce6667855`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:23.409768 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a132b8f5-1805-4aff-b509-38a9db8b1206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:23.510631 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1017/2031] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:23.522554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00132e-24bd-46dd-8fbe-bbe0a053891c service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:23.594351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00132e-24bd-46dd-8fbe-bbe0a053891c service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:23.598161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27959dba-7707-43b1-a857-4805780918d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:23.598844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27959dba-7707-43b1-a857-4805780918d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:23.599213 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27959dba-7707-43b1-a857-4805780918d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:23.599428 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27959dba-7707-43b1-a857-4805780918d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:23.610090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27959dba-7707-43b1-a857-4805780918d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:23.611599 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27959dba-7707-43b1-a857-4805780918d6 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:23.623215 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1015/2032] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:25.736556 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa5c0e-7a3c-4a10-babc-727afb9187d6 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:25.737593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa5c0e-7a3c-4a10-babc-727afb9187d6 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:25.738072 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa5c0e-7a3c-4a10-babc-727afb9187d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:25.757079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:25.769748 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-e7fa5c0e-7a3c-4a10-babc-727afb9187d6 None None] MFA Rules not processed for user `8ba8498f6fbc40c18cadbb4ce6667855`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:25.772615 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e7fa5c0e-7a3c-4a10-babc-727afb9187d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:25.820719 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1018/2033] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:26.133391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27959dba-7707-43b1-a857-4805780918d6 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:26.136353 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-27959dba-7707-43b1-a857-4805780918d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:26.156170 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27959dba-7707-43b1-a857-4805780918d6 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:26.163411 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ddc1a-37c7-4d5d-8cb0-9ecaedf1ee02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:26.164623 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ddc1a-37c7-4d5d-8cb0-9ecaedf1ee02 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:26.165254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ddc1a-37c7-4d5d-8cb0-9ecaedf1ee02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:26.165687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ddc1a-37c7-4d5d-8cb0-9ecaedf1ee02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:26.168014 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-696ddc1a-37c7-4d5d-8cb0-9ecaedf1ee02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:26.290499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:26.311333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ddc1a-37c7-4d5d-8cb0-9ecaedf1ee02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:26.314512 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696ddc1a-37c7-4d5d-8cb0-9ecaedf1ee02 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:26.328816 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1016/2034] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:27.987551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa5c0e-7a3c-4a10-babc-727afb9187d6 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:28.005536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa5c0e-7a3c-4a10-babc-727afb9187d6 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:28.017806 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451b376-7e69-4f08-8d59-a4234f0e06b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:28.019431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451b376-7e69-4f08-8d59-a4234f0e06b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:28.019913 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451b376-7e69-4f08-8d59-a4234f0e06b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:28.020335 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451b376-7e69-4f08-8d59-a4234f0e06b8 admin admin] PATH_INFO: `/v3/users/e7b8b164d575415795203ff63a9398cd` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:28.032095 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451b376-7e69-4f08-8d59-a4234f0e06b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7b8b164d575415795203ff63a9398cd)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:28.034444 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451b376-7e69-4f08-8d59-a4234f0e06b8 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:28.341546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7451b376-7e69-4f08-8d59-a4234f0e06b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7b8b164d575415795203ff63a9398cd'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:28.440784 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7451b376-7e69-4f08-8d59-a4234f0e06b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7b8b164d575415795203ff63a9398cd'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:28.451950 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7451b376-7e69-4f08-8d59-a4234f0e06b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7b8b164d575415795203ff63a9398cd'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:28.470693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7451b376-7e69-4f08-8d59-a4234f0e06b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7b8b164d575415795203ff63a9398cd'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:28.471962 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7451b376-7e69-4f08-8d59-a4234f0e06b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7b8b164d575415795203ff63a9398cd'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:28.480036 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1019/2035] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:12:27 2022] DELETE /identity/v3/users/e7b8b164d575415795203ff63a9398cd => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:12:28.485969 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959ed6e-d9f0-4380-bcc9-459caf1500b0 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:28.486526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959ed6e-d9f0-4380-bcc9-459caf1500b0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:28.486843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959ed6e-d9f0-4380-bcc9-459caf1500b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:28.509444 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c959ed6e-d9f0-4380-bcc9-459caf1500b0 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:28.511015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c959ed6e-d9f0-4380-bcc9-459caf1500b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:28.607981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1017/2036] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:28.620030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451b376-7e69-4f08-8d59-a4234f0e06b8 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:28.622034 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7451b376-7e69-4f08-8d59-a4234f0e06b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:28.737086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451b376-7e69-4f08-8d59-a4234f0e06b8 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:28.737086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa9198-676c-4953-b2c3-ab8e423775a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:28.737086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa9198-676c-4953-b2c3-ab8e423775a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:28.737515 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa9198-676c-4953-b2c3-ab8e423775a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:28.738015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa9198-676c-4953-b2c3-ab8e423775a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:28.751235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa9198-676c-4953-b2c3-ab8e423775a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:28.752714 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa9198-676c-4953-b2c3-ab8e423775a0 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:28.764010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1020/2037] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:29.569947 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959ed6e-d9f0-4380-bcc9-459caf1500b0 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:29.572936 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c959ed6e-d9f0-4380-bcc9-459caf1500b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:29.641806 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa9198-676c-4953-b2c3-ab8e423775a0 service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:29.643425 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-38fa9198-676c-4953-b2c3-ab8e423775a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:29.651143 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959ed6e-d9f0-4380-bcc9-459caf1500b0 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:29.656541 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6734614-05ad-4668-9bbb-666c04191470 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:29.657736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6734614-05ad-4668-9bbb-666c04191470 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:29.657839 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6734614-05ad-4668-9bbb-666c04191470 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:29.658292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6734614-05ad-4668-9bbb-666c04191470 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:29.660603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a6734614-05ad-4668-9bbb-666c04191470 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:29.704137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa9198-676c-4953-b2c3-ab8e423775a0 service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:29.714983 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18764f40-4b1f-4e90-a493-507c4b562b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:29.716682 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18764f40-4b1f-4e90-a493-507c4b562b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:29.717909 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18764f40-4b1f-4e90-a493-507c4b562b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:29.718541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18764f40-4b1f-4e90-a493-507c4b562b73 admin admin] PATH_INFO: `/v3/projects/d566f74e0d83492aad3bfcee2f4c7d67` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:29.727115 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18764f40-4b1f-4e90-a493-507c4b562b73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d566f74e0d83492aad3bfcee2f4c7d67)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:29.727115 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18764f40-4b1f-4e90-a493-507c4b562b73 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:29.779811 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6734614-05ad-4668-9bbb-666c04191470 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:29.782095 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6734614-05ad-4668-9bbb-666c04191470 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:29.795822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1018/2038] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:29 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:29.895551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-18764f40-4b1f-4e90-a493-507c4b562b73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd566f74e0d83492aad3bfcee2f4c7d67', 'request_id': 'req-18764f40-4b1f-4e90-a493-507c4b562b73'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:29.913320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-18764f40-4b1f-4e90-a493-507c4b562b73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd566f74e0d83492aad3bfcee2f4c7d67', 'request_id': 'req-18764f40-4b1f-4e90-a493-507c4b562b73'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:29.921079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-18764f40-4b1f-4e90-a493-507c4b562b73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd566f74e0d83492aad3bfcee2f4c7d67', 'request_id': 'req-18764f40-4b1f-4e90-a493-507c4b562b73'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:30.061219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:30.068408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-18764f40-4b1f-4e90-a493-507c4b562b73 admin admin] The token cache is being invalidate because project d566f74e0d83492aad3bfcee2f4c7d67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:12:30.068751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-18764f40-4b1f-4e90-a493-507c4b562b73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:30.078925 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-18764f40-4b1f-4e90-a493-507c4b562b73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:30.085431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1021/2039] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:12:29 2022] DELETE /identity/v3/projects/d566f74e0d83492aad3bfcee2f4c7d67 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:30.086769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6734614-05ad-4668-9bbb-666c04191470 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:30.100002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6734614-05ad-4668-9bbb-666c04191470 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:30.167820 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:30.237216 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15c429-89e4-4564-80d4-0d89055e776b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:30.238404 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15c429-89e4-4564-80d4-0d89055e776b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:30.239477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15c429-89e4-4564-80d4-0d89055e776b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:30.239477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15c429-89e4-4564-80d4-0d89055e776b admin admin] PATH_INFO: `/v3/users/8e07996ae3b04ff9a891387e1940dff0` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:30.241480 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce15c429-89e4-4564-80d4-0d89055e776b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e07996ae3b04ff9a891387e1940dff0)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:30.244288 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce15c429-89e4-4564-80d4-0d89055e776b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:30.564154 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ce15c429-89e4-4564-80d4-0d89055e776b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e07996ae3b04ff9a891387e1940dff0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:30.659258 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ce15c429-89e4-4564-80d4-0d89055e776b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e07996ae3b04ff9a891387e1940dff0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:30.660283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ce15c429-89e4-4564-80d4-0d89055e776b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e07996ae3b04ff9a891387e1940dff0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:30.661229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ce15c429-89e4-4564-80d4-0d89055e776b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e07996ae3b04ff9a891387e1940dff0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:30.691389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ce15c429-89e4-4564-80d4-0d89055e776b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e07996ae3b04ff9a891387e1940dff0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:30.740009 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1019/2040] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:12:30 2022] DELETE /identity/v3/users/8e07996ae3b04ff9a891387e1940dff0 => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:12:31.305691 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19306d5e-1273-4c46-90ee-55553fa23e6f None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:31.313676 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19306d5e-1273-4c46-90ee-55553fa23e6f None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:31.313676 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19306d5e-1273-4c46-90ee-55553fa23e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:31.354388 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-19306d5e-1273-4c46-90ee-55553fa23e6f None None] MFA Rules not processed for user `4cadd4bc4d36437484f495d0e49d9881`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:31.355938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-19306d5e-1273-4c46-90ee-55553fa23e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:31.443367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1022/2041] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:31.461542 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15c429-89e4-4564-80d4-0d89055e776b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:31.464681 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ce15c429-89e4-4564-80d4-0d89055e776b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:31.566940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15c429-89e4-4564-80d4-0d89055e776b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:31.576958 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02fa7b-743d-41ce-869d-775f438531f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:31.578414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02fa7b-743d-41ce-869d-775f438531f0 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:31.578587 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02fa7b-743d-41ce-869d-775f438531f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:31.579142 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02fa7b-743d-41ce-869d-775f438531f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:31.602661 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02fa7b-743d-41ce-869d-775f438531f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:31.604897 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02fa7b-743d-41ce-869d-775f438531f0 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:31.635099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1020/2042] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:12:32.011430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19306d5e-1273-4c46-90ee-55553fa23e6f None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:32.013148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-19306d5e-1273-4c46-90ee-55553fa23e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:32.145598 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19306d5e-1273-4c46-90ee-55553fa23e6f None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:32.152677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b16133-3561-445e-b588-c9a7c1f2c607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:32.154765 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b16133-3561-445e-b588-c9a7c1f2c607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:32.154765 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b16133-3561-445e-b588-c9a7c1f2c607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:32.157748 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b16133-3561-445e-b588-c9a7c1f2c607 admin admin] PATH_INFO: `/v3/projects/795f7adf639d491f869644b7d5fb2748` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:32.160430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b16133-3561-445e-b588-c9a7c1f2c607 admin admin] RBAC: Authorizing `identity:delete_project(project_id=795f7adf639d491f869644b7d5fb2748)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:32.162746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b16133-3561-445e-b588-c9a7c1f2c607 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:32.321172 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-57b16133-3561-445e-b588-c9a7c1f2c607 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '795f7adf639d491f869644b7d5fb2748', 'request_id': 'req-57b16133-3561-445e-b588-c9a7c1f2c607'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:32.336831 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-57b16133-3561-445e-b588-c9a7c1f2c607 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '795f7adf639d491f869644b7d5fb2748', 'request_id': 'req-57b16133-3561-445e-b588-c9a7c1f2c607'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:32.347571 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-57b16133-3561-445e-b588-c9a7c1f2c607 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '795f7adf639d491f869644b7d5fb2748', 'request_id': 'req-57b16133-3561-445e-b588-c9a7c1f2c607'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:32.435064 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-57b16133-3561-445e-b588-c9a7c1f2c607 admin admin] The token cache is being invalidate because project 795f7adf639d491f869644b7d5fb2748 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:12:32.435364 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-57b16133-3561-445e-b588-c9a7c1f2c607 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:32.437949 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-57b16133-3561-445e-b588-c9a7c1f2c607 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:32.442945 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1023/2043] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:12:32 2022] DELETE /identity/v3/projects/795f7adf639d491f869644b7d5fb2748 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:32.445322 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbecd6-8d96-4e44-947a-31ea8c283965 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:32.445322 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbecd6-8d96-4e44-947a-31ea8c283965 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:32.445322 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbecd6-8d96-4e44-947a-31ea8c283965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:32.465131 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-75cbecd6-8d96-4e44-947a-31ea8c283965 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:32.466377 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-75cbecd6-8d96-4e44-947a-31ea8c283965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:32.509041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:32.537332 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1021/2044] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:12:32.545402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073fb46-2d2c-4f05-8a02-d576311204e8 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:32.548996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073fb46-2d2c-4f05-8a02-d576311204e8 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:32.548996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073fb46-2d2c-4f05-8a02-d576311204e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:32.569690 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-9073fb46-2d2c-4f05-8a02-d576311204e8 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:32.571532 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-9073fb46-2d2c-4f05-8a02-d576311204e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:32.583924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1024/2045] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:32.594539 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbecd6-8d96-4e44-947a-31ea8c283965 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:32.607765 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbecd6-8d96-4e44-947a-31ea8c283965 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:32.614940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299daa4-8a56-445a-81b7-1a4f99380dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:32.616959 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299daa4-8a56-445a-81b7-1a4f99380dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:32.617690 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299daa4-8a56-445a-81b7-1a4f99380dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:32.618823 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299daa4-8a56-445a-81b7-1a4f99380dc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:32.619698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299daa4-8a56-445a-81b7-1a4f99380dc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:12:32.620204 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299daa4-8a56-445a-81b7-1a4f99380dc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:32.622029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299daa4-8a56-445a-81b7-1a4f99380dc1 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:32.636847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1022/2046] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:12:32.644032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073fb46-2d2c-4f05-8a02-d576311204e8 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:32.657807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073fb46-2d2c-4f05-8a02-d576311204e8 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:32.663348 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8669f7-d54f-47ef-844d-5677f5384c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:32.664034 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8669f7-d54f-47ef-844d-5677f5384c7d admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:32.664349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8669f7-d54f-47ef-844d-5677f5384c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:32.664597 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8669f7-d54f-47ef-844d-5677f5384c7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:32.665333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8669f7-d54f-47ef-844d-5677f5384c7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:32.667647 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8669f7-d54f-47ef-844d-5677f5384c7d admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:32.677542 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:32.791856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1025/2047] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:12:32 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:12:32.793245 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299daa4-8a56-445a-81b7-1a4f99380dc1 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:32.813820 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299daa4-8a56-445a-81b7-1a4f99380dc1 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:32.883924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb132d-d917-4158-94ed-c091a6105c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:32.884894 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb132d-d917-4158-94ed-c091a6105c68 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:32.885479 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb132d-d917-4158-94ed-c091a6105c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:32.885988 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb132d-d917-4158-94ed-c091a6105c68 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:32.887858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb132d-d917-4158-94ed-c091a6105c68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:32.890764 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb132d-d917-4158-94ed-c091a6105c68 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:33.047144 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1023/2048] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:12:32 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:33.048561 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8669f7-d54f-47ef-844d-5677f5384c7d admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:33.064127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8669f7-d54f-47ef-844d-5677f5384c7d admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:33.072445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036ff45-a1fb-4b68-9b33-afd4f2846527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:33.073315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036ff45-a1fb-4b68-9b33-afd4f2846527 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:33.073628 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036ff45-a1fb-4b68-9b33-afd4f2846527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:33.073913 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036ff45-a1fb-4b68-9b33-afd4f2846527 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:33.074929 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0036ff45-a1fb-4b68-9b33-afd4f2846527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:33.077515 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0036ff45-a1fb-4b68-9b33-afd4f2846527 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:33.100121 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1026/2049] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:33.106699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb132d-d917-4158-94ed-c091a6105c68 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:33.125036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb132d-d917-4158-94ed-c091a6105c68 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:33.132233 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2e319-9474-42e9-b471-72c854de7011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:33.133385 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2e319-9474-42e9-b471-72c854de7011 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:33.133978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2e319-9474-42e9-b471-72c854de7011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:33.134591 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2e319-9474-42e9-b471-72c854de7011 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:33.135781 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2e319-9474-42e9-b471-72c854de7011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:33.139591 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2e319-9474-42e9-b471-72c854de7011 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:33.154556 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1024/2050] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:33.160270 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036ff45-a1fb-4b68-9b33-afd4f2846527 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:33.190708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036ff45-a1fb-4b68-9b33-afd4f2846527 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:33.200411 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b031c-61f2-49d4-9398-d00f1df3c768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:33.202102 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b031c-61f2-49d4-9398-d00f1df3c768 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:33.203181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b031c-61f2-49d4-9398-d00f1df3c768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:33.205200 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b031c-61f2-49d4-9398-d00f1df3c768 admin admin] PATH_INFO: `/v3/projects/7247dd612f6d480485a5857593c36831/users/b7a1047afa4940fa969e0ca5fc19dbb8/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:33.241471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b031c-61f2-49d4-9398-d00f1df3c768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7247dd612f6d480485a5857593c36831, user_id=b7a1047afa4940fa969e0ca5fc19dbb8, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:33.248463 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b031c-61f2-49d4-9398-d00f1df3c768 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:33.380840 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1027/2051] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:33 2022] PUT /identity/v3/projects/7247dd612f6d480485a5857593c36831/users/b7a1047afa4940fa969e0ca5fc19dbb8/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:33.400509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b590b76-2261-44d4-b87c-5a64644f3e5f None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:33.400802 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b590b76-2261-44d4-b87c-5a64644f3e5f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:33.401098 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b590b76-2261-44d4-b87c-5a64644f3e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:33.477177 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-9b590b76-2261-44d4-b87c-5a64644f3e5f None None] MFA Rules not processed for user `b7a1047afa4940fa969e0ca5fc19dbb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:33.481986 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9b590b76-2261-44d4-b87c-5a64644f3e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:33.714853 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b031c-61f2-49d4-9398-d00f1df3c768 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:33.716335 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1025/2052] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:33.882626 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b031c-61f2-49d4-9398-d00f1df3c768 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:33.891490 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b9306-e4f5-4e25-aa21-ee4b0c12106b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:33.892221 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b9306-e4f5-4e25-aa21-ee4b0c12106b admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:33.892845 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b9306-e4f5-4e25-aa21-ee4b0c12106b admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:33.893198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b9306-e4f5-4e25-aa21-ee4b0c12106b admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:33.894036 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83b9306-e4f5-4e25-aa21-ee4b0c12106b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:33.896792 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83b9306-e4f5-4e25-aa21-ee4b0c12106b admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:34.059266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1028/2053] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:12:33 2022] POST /identity/v3/projects => generated 385 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:12:34.063651 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b590b76-2261-44d4-b87c-5a64644f3e5f None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:34.083661 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b590b76-2261-44d4-b87c-5a64644f3e5f None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:34.196397 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:34.302745 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878337d-4c7c-4cc3-935b-b00088a79851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:34.303718 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878337d-4c7c-4cc3-935b-b00088a79851 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:34.303932 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878337d-4c7c-4cc3-935b-b00088a79851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:34.304426 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878337d-4c7c-4cc3-935b-b00088a79851 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:34.305207 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878337d-4c7c-4cc3-935b-b00088a79851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:34.307410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878337d-4c7c-4cc3-935b-b00088a79851 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:34.436304 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1026/2054] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:12:34 2022] POST /identity/v3/users => generated 299 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:12:34.441672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b9306-e4f5-4e25-aa21-ee4b0c12106b admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:34.465962 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b9306-e4f5-4e25-aa21-ee4b0c12106b admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:34.473422 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95545f5-8785-4c87-9aa4-94c34fbad878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:34.474333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95545f5-8785-4c87-9aa4-94c34fbad878 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:34.475283 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95545f5-8785-4c87-9aa4-94c34fbad878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:34.475827 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95545f5-8785-4c87-9aa4-94c34fbad878 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:34.477161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95545f5-8785-4c87-9aa4-94c34fbad878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:34.479676 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95545f5-8785-4c87-9aa4-94c34fbad878 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:34.504070 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1029/2055] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:34.508102 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878337d-4c7c-4cc3-935b-b00088a79851 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:34.525886 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878337d-4c7c-4cc3-935b-b00088a79851 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:34.537476 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42945631-f199-4b65-bfc2-cc89807d6e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:34.538683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42945631-f199-4b65-bfc2-cc89807d6e6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:34.539183 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42945631-f199-4b65-bfc2-cc89807d6e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:34.539709 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42945631-f199-4b65-bfc2-cc89807d6e6b admin admin] PATH_INFO: `/v3/projects/d904b5a36600462abbc6e7d4b8f69d8b/users/6c72d8fe4dfd40cfa26c70662d256019/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:34.566971 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42945631-f199-4b65-bfc2-cc89807d6e6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d904b5a36600462abbc6e7d4b8f69d8b, user_id=6c72d8fe4dfd40cfa26c70662d256019, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:34.570250 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42945631-f199-4b65-bfc2-cc89807d6e6b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:34.688492 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1027/2056] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:34 2022] PUT /identity/v3/projects/d904b5a36600462abbc6e7d4b8f69d8b/users/6c72d8fe4dfd40cfa26c70662d256019/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:34.691664 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95545f5-8785-4c87-9aa4-94c34fbad878 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:34.705349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95545f5-8785-4c87-9aa4-94c34fbad878 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:34.806996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8152d9-45bc-4c00-b4aa-a280d5e28c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:34.810573 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8152d9-45bc-4c00-b4aa-a280d5e28c13 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:34.811122 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8152d9-45bc-4c00-b4aa-a280d5e28c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:34.812665 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8152d9-45bc-4c00-b4aa-a280d5e28c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:34.813995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8152d9-45bc-4c00-b4aa-a280d5e28c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:34.818063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8152d9-45bc-4c00-b4aa-a280d5e28c13 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:34.850690 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1030/2057] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:12:34.857331 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42945631-f199-4b65-bfc2-cc89807d6e6b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:34.872997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42945631-f199-4b65-bfc2-cc89807d6e6b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:34.881853 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030c3cf-f3c1-4161-9da1-c18d1904a8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:34.883474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030c3cf-f3c1-4161-9da1-c18d1904a8fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:34.884006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030c3cf-f3c1-4161-9da1-c18d1904a8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:34.884570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030c3cf-f3c1-4161-9da1-c18d1904a8fa admin admin] PATH_INFO: `/v3/projects/d904b5a36600462abbc6e7d4b8f69d8b/users/6c72d8fe4dfd40cfa26c70662d256019/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:34.891480 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030c3cf-f3c1-4161-9da1-c18d1904a8fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d904b5a36600462abbc6e7d4b8f69d8b, user_id=6c72d8fe4dfd40cfa26c70662d256019, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:34.894794 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030c3cf-f3c1-4161-9da1-c18d1904a8fa admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:34.912000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1028/2058] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:34 2022] PUT /identity/v3/projects/d904b5a36600462abbc6e7d4b8f69d8b/users/6c72d8fe4dfd40cfa26c70662d256019/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:12:34.932165 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67f823-c28f-4c9a-a07d-79a00e0ef139 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:34.932629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67f823-c28f-4c9a-a07d-79a00e0ef139 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:34.933145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67f823-c28f-4c9a-a07d-79a00e0ef139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:34.989947 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-1b67f823-c28f-4c9a-a07d-79a00e0ef139 None None] MFA Rules not processed for user `6c72d8fe4dfd40cfa26c70662d256019`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:34.992259 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-1b67f823-c28f-4c9a-a07d-79a00e0ef139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:35.020169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:35.200802 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1031/2059] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:35.230660 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030c3cf-f3c1-4161-9da1-c18d1904a8fa admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:35.244914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b030c3cf-f3c1-4161-9da1-c18d1904a8fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:35.391895 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030c3cf-f3c1-4161-9da1-c18d1904a8fa admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:35.397011 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afcac2-32e0-4bcb-bb57-9183267ce2d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:35.398793 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afcac2-32e0-4bcb-bb57-9183267ce2d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:35.399158 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afcac2-32e0-4bcb-bb57-9183267ce2d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:35.399460 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afcac2-32e0-4bcb-bb57-9183267ce2d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:35.528374 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41afcac2-32e0-4bcb-bb57-9183267ce2d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:35.531589 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41afcac2-32e0-4bcb-bb57-9183267ce2d2 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:35.554081 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1029/2060] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:36.986438 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24032e41-f4f4-4926-97b3-3eba85115343 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:36.987180 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24032e41-f4f4-4926-97b3-3eba85115343 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:36.987555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24032e41-f4f4-4926-97b3-3eba85115343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:37.025489 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-24032e41-f4f4-4926-97b3-3eba85115343 None None] MFA Rules not processed for user `4cadd4bc4d36437484f495d0e49d9881`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:37.029313 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-24032e41-f4f4-4926-97b3-3eba85115343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:37.141531 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1032/2061] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:38.086256 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afcac2-32e0-4bcb-bb57-9183267ce2d2 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:38.090660 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-41afcac2-32e0-4bcb-bb57-9183267ce2d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:38.196202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:38.239848 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afcac2-32e0-4bcb-bb57-9183267ce2d2 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:38.250718 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704426b-0b5c-4e3d-a6ce-1aa1e35826ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:38.252426 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704426b-0b5c-4e3d-a6ce-1aa1e35826ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:38.252998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704426b-0b5c-4e3d-a6ce-1aa1e35826ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:38.253794 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704426b-0b5c-4e3d-a6ce-1aa1e35826ab admin admin] PATH_INFO: `/v3/users/eb220b76c35b46ccb4bdaa617c574880` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:38.259026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704426b-0b5c-4e3d-a6ce-1aa1e35826ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb220b76c35b46ccb4bdaa617c574880)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:38.263729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704426b-0b5c-4e3d-a6ce-1aa1e35826ab admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:38.623117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5704426b-0b5c-4e3d-a6ce-1aa1e35826ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb220b76c35b46ccb4bdaa617c574880'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:38.760495 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5704426b-0b5c-4e3d-a6ce-1aa1e35826ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb220b76c35b46ccb4bdaa617c574880'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:38.761390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5704426b-0b5c-4e3d-a6ce-1aa1e35826ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb220b76c35b46ccb4bdaa617c574880'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:38.762036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5704426b-0b5c-4e3d-a6ce-1aa1e35826ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb220b76c35b46ccb4bdaa617c574880'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:38.772062 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5704426b-0b5c-4e3d-a6ce-1aa1e35826ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb220b76c35b46ccb4bdaa617c574880'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:38.813867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1030/2062] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:12:38 2022] DELETE /identity/v3/users/eb220b76c35b46ccb4bdaa617c574880 => generated 0 bytes in 729 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:38.819839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f7e15-44a2-402c-98f8-49703ed764d9 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:38.820845 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f7e15-44a2-402c-98f8-49703ed764d9 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:38.821328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f7e15-44a2-402c-98f8-49703ed764d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:38.857281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-277f7e15-44a2-402c-98f8-49703ed764d9 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:38.858653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-277f7e15-44a2-402c-98f8-49703ed764d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:38.987395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1033/2063] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:12:39.001884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704426b-0b5c-4e3d-a6ce-1aa1e35826ab admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:39.004500 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5704426b-0b5c-4e3d-a6ce-1aa1e35826ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:39.103027 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704426b-0b5c-4e3d-a6ce-1aa1e35826ab admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:39.107446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c967f10-4d08-40a0-94d9-57e86bed73f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:39.108054 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c967f10-4d08-40a0-94d9-57e86bed73f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:39.108409 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c967f10-4d08-40a0-94d9-57e86bed73f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:39.108652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c967f10-4d08-40a0-94d9-57e86bed73f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:39.121187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c967f10-4d08-40a0-94d9-57e86bed73f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:39.122662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c967f10-4d08-40a0-94d9-57e86bed73f7 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:39.135036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1031/2064] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:39.839462 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f7e15-44a2-402c-98f8-49703ed764d9 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:39.841877 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-277f7e15-44a2-402c-98f8-49703ed764d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:39.866205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f7e15-44a2-402c-98f8-49703ed764d9 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:39.882208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1329d5-a805-4974-ae0a-92b05f551cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:39.883801 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1329d5-a805-4974-ae0a-92b05f551cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:39.884330 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1329d5-a805-4974-ae0a-92b05f551cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:39.884834 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1329d5-a805-4974-ae0a-92b05f551cc5 admin admin] PATH_INFO: `/v3/projects/24aacc1c4e2943e39ab19ac312445d44` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:39.887320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1329d5-a805-4974-ae0a-92b05f551cc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24aacc1c4e2943e39ab19ac312445d44)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:39.892373 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1329d5-a805-4974-ae0a-92b05f551cc5 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:40.023110 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-3e1329d5-a805-4974-ae0a-92b05f551cc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24aacc1c4e2943e39ab19ac312445d44', 'request_id': 'req-3e1329d5-a805-4974-ae0a-92b05f551cc5'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:40.035364 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:40.035364 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-3e1329d5-a805-4974-ae0a-92b05f551cc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24aacc1c4e2943e39ab19ac312445d44', 'request_id': 'req-3e1329d5-a805-4974-ae0a-92b05f551cc5'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:40.043808 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-3e1329d5-a805-4974-ae0a-92b05f551cc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24aacc1c4e2943e39ab19ac312445d44', 'request_id': 'req-3e1329d5-a805-4974-ae0a-92b05f551cc5'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:40.144588 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-3e1329d5-a805-4974-ae0a-92b05f551cc5 admin admin] The token cache is being invalidate because project 24aacc1c4e2943e39ab19ac312445d44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:12:40.144892 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-3e1329d5-a805-4974-ae0a-92b05f551cc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:40.145646 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-3e1329d5-a805-4974-ae0a-92b05f551cc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:40.147447 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1034/2065] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:12:39 2022] DELETE /identity/v3/projects/24aacc1c4e2943e39ab19ac312445d44 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:40.153864 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c967f10-4d08-40a0-94d9-57e86bed73f7 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:40.167107 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c967f10-4d08-40a0-94d9-57e86bed73f7 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:40.253607 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73f906-818b-4305-8586-99e0f630b181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:40.254544 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73f906-818b-4305-8586-99e0f630b181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:40.254820 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73f906-818b-4305-8586-99e0f630b181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:40.255135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73f906-818b-4305-8586-99e0f630b181 admin admin] PATH_INFO: `/v3/users/f98d98ec55514c14839e4761509d0538` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:40.256448 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73f906-818b-4305-8586-99e0f630b181 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f98d98ec55514c14839e4761509d0538)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:40.258475 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73f906-818b-4305-8586-99e0f630b181 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:40.264406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:40.490810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3f73f906-818b-4305-8586-99e0f630b181 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f98d98ec55514c14839e4761509d0538'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:40.573797 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3f73f906-818b-4305-8586-99e0f630b181 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f98d98ec55514c14839e4761509d0538'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:40.575121 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3f73f906-818b-4305-8586-99e0f630b181 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f98d98ec55514c14839e4761509d0538'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:40.576859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3f73f906-818b-4305-8586-99e0f630b181 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f98d98ec55514c14839e4761509d0538'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:40.591483 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3f73f906-818b-4305-8586-99e0f630b181 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f98d98ec55514c14839e4761509d0538'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:40.629632 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1032/2066] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:12:40 2022] DELETE /identity/v3/users/f98d98ec55514c14839e4761509d0538 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:41.385664 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1329d5-a805-4974-ae0a-92b05f551cc5 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:41.391184 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3e1329d5-a805-4974-ae0a-92b05f551cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:41.415101 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1329d5-a805-4974-ae0a-92b05f551cc5 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:41.503515 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6370f65-6ab6-4984-a27f-4ca29ed02ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:41.504407 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6370f65-6ab6-4984-a27f-4ca29ed02ac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:41.504686 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6370f65-6ab6-4984-a27f-4ca29ed02ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:41.504961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6370f65-6ab6-4984-a27f-4ca29ed02ac6 admin admin] PATH_INFO: `/v3/projects/12c555244cfc4599a422b00782cebb37` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:41.506281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6370f65-6ab6-4984-a27f-4ca29ed02ac6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12c555244cfc4599a422b00782cebb37)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:41.508386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6370f65-6ab6-4984-a27f-4ca29ed02ac6 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:41.684726 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e6370f65-6ab6-4984-a27f-4ca29ed02ac6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12c555244cfc4599a422b00782cebb37', 'request_id': 'req-e6370f65-6ab6-4984-a27f-4ca29ed02ac6'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:41.697608 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e6370f65-6ab6-4984-a27f-4ca29ed02ac6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12c555244cfc4599a422b00782cebb37', 'request_id': 'req-e6370f65-6ab6-4984-a27f-4ca29ed02ac6'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:41.712265 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e6370f65-6ab6-4984-a27f-4ca29ed02ac6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12c555244cfc4599a422b00782cebb37', 'request_id': 'req-e6370f65-6ab6-4984-a27f-4ca29ed02ac6'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:41.840898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e6370f65-6ab6-4984-a27f-4ca29ed02ac6 admin admin] The token cache is being invalidate because project 12c555244cfc4599a422b00782cebb37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:12:41.841289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e6370f65-6ab6-4984-a27f-4ca29ed02ac6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:41.842082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e6370f65-6ab6-4984-a27f-4ca29ed02ac6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:41.845374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1035/2067] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:12:41 2022] DELETE /identity/v3/projects/12c555244cfc4599a422b00782cebb37 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:41.849327 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73f906-818b-4305-8586-99e0f630b181 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:41.856754 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3f73f906-818b-4305-8586-99e0f630b181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:41.865468 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73f906-818b-4305-8586-99e0f630b181 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:41.946749 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd606838-1f7f-49e9-a0d1-83e034faf3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:41.947690 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd606838-1f7f-49e9-a0d1-83e034faf3a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:41.947972 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd606838-1f7f-49e9-a0d1-83e034faf3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:41.948255 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd606838-1f7f-49e9-a0d1-83e034faf3a6 admin admin] PATH_INFO: `/v3/users/af117bf9ef5b4e3b87deb2f896cd01cc` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:41.949675 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd606838-1f7f-49e9-a0d1-83e034faf3a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af117bf9ef5b4e3b87deb2f896cd01cc)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:41.951515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd606838-1f7f-49e9-a0d1-83e034faf3a6 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:42.057913 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6370f65-6ab6-4984-a27f-4ca29ed02ac6 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:42.060750 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e6370f65-6ab6-4984-a27f-4ca29ed02ac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:42.224073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6370f65-6ab6-4984-a27f-4ca29ed02ac6 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:42.233104 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d166d66-1af8-40ee-ae52-07f0217bf564 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:42.234213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d166d66-1af8-40ee-ae52-07f0217bf564 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:42.234701 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d166d66-1af8-40ee-ae52-07f0217bf564 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:42.237847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d166d66-1af8-40ee-ae52-07f0217bf564 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:42.247114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3d166d66-1af8-40ee-ae52-07f0217bf564 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:42.270999 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fd606838-1f7f-49e9-a0d1-83e034faf3a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af117bf9ef5b4e3b87deb2f896cd01cc'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:42.313232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:42.372229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fd606838-1f7f-49e9-a0d1-83e034faf3a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af117bf9ef5b4e3b87deb2f896cd01cc'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:42.372880 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fd606838-1f7f-49e9-a0d1-83e034faf3a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af117bf9ef5b4e3b87deb2f896cd01cc'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:42.373511 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fd606838-1f7f-49e9-a0d1-83e034faf3a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af117bf9ef5b4e3b87deb2f896cd01cc'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:42.387254 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d166d66-1af8-40ee-ae52-07f0217bf564 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:42.388095 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fd606838-1f7f-49e9-a0d1-83e034faf3a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af117bf9ef5b4e3b87deb2f896cd01cc'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:42.390521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d166d66-1af8-40ee-ae52-07f0217bf564 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:42.409941 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1036/2068] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:42.411556 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d166d66-1af8-40ee-ae52-07f0217bf564 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:42.412624 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1033/2069] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:12:41 2022] DELETE /identity/v3/users/af117bf9ef5b4e3b87deb2f896cd01cc => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:12:42.414150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3d166d66-1af8-40ee-ae52-07f0217bf564 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:42.441249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d166d66-1af8-40ee-ae52-07f0217bf564 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:42.563432 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415a4c6-d734-424b-a1a6-64aa68c5c341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:42.565230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415a4c6-d734-424b-a1a6-64aa68c5c341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:42.566066 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415a4c6-d734-424b-a1a6-64aa68c5c341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:42.568002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415a4c6-d734-424b-a1a6-64aa68c5c341 admin admin] PATH_INFO: `/v3/users/116c4f95c0a64ad190f32c64500f7e78` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:42.570952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415a4c6-d734-424b-a1a6-64aa68c5c341 admin admin] RBAC: Authorizing `identity:delete_user(user_id=116c4f95c0a64ad190f32c64500f7e78)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:42.575320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415a4c6-d734-424b-a1a6-64aa68c5c341 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:43.044241 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-4415a4c6-d734-424b-a1a6-64aa68c5c341 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '116c4f95c0a64ad190f32c64500f7e78'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:43.132813 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-4415a4c6-d734-424b-a1a6-64aa68c5c341 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '116c4f95c0a64ad190f32c64500f7e78'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:43.147774 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-4415a4c6-d734-424b-a1a6-64aa68c5c341 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '116c4f95c0a64ad190f32c64500f7e78'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:43.178477 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-4415a4c6-d734-424b-a1a6-64aa68c5c341 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '116c4f95c0a64ad190f32c64500f7e78'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:43.181128 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-4415a4c6-d734-424b-a1a6-64aa68c5c341 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '116c4f95c0a64ad190f32c64500f7e78'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:43.199770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1037/2070] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:12:42 2022] DELETE /identity/v3/users/116c4f95c0a64ad190f32c64500f7e78 => generated 0 bytes in 788 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:43.207032 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326710d1-9c07-4669-8239-8730fd9487a8 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:43.207557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326710d1-9c07-4669-8239-8730fd9487a8 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:43.208786 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326710d1-9c07-4669-8239-8730fd9487a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:43.225795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:43.246298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-326710d1-9c07-4669-8239-8730fd9487a8 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:43.250165 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-326710d1-9c07-4669-8239-8730fd9487a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:43.330372 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415a4c6-d734-424b-a1a6-64aa68c5c341 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:43.331763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-4415a4c6-d734-424b-a1a6-64aa68c5c341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:43.350696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1034/2071] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:43.363081 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326710d1-9c07-4669-8239-8730fd9487a8 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:43.387327 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:43.393859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-326710d1-9c07-4669-8239-8730fd9487a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:43.445237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415a4c6-d734-424b-a1a6-64aa68c5c341 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:43.463176 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0cbf0e-0d17-4cc7-99b2-cc5526cb0ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:43.464400 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cbf0e-0d17-4cc7-99b2-cc5526cb0ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:43.464981 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cbf0e-0d17-4cc7-99b2-cc5526cb0ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:43.465571 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0cbf0e-0d17-4cc7-99b2-cc5526cb0ea1 admin admin] PATH_INFO: `/v3/projects/11a233b00bfb4a10acc46f21156d1baa` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:43.467219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0cbf0e-0d17-4cc7-99b2-cc5526cb0ea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11a233b00bfb4a10acc46f21156d1baa)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:43.470051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0cbf0e-0d17-4cc7-99b2-cc5526cb0ea1 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:43.505681 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326710d1-9c07-4669-8239-8730fd9487a8 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:43.513496 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1c463-2e4c-47eb-ba17-f8246227264e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:43.514470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1c463-2e4c-47eb-ba17-f8246227264e service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:43.516236 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1c463-2e4c-47eb-ba17-f8246227264e service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:43.517497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1c463-2e4c-47eb-ba17-f8246227264e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:43.560708 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca1c463-2e4c-47eb-ba17-f8246227264e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:43.562249 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca1c463-2e4c-47eb-ba17-f8246227264e service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:43.575162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1035/2072] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:43.619463 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-bd0cbf0e-0d17-4cc7-99b2-cc5526cb0ea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11a233b00bfb4a10acc46f21156d1baa', 'request_id': 'req-bd0cbf0e-0d17-4cc7-99b2-cc5526cb0ea1'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:43.627867 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-bd0cbf0e-0d17-4cc7-99b2-cc5526cb0ea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11a233b00bfb4a10acc46f21156d1baa', 'request_id': 'req-bd0cbf0e-0d17-4cc7-99b2-cc5526cb0ea1'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:43.636037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-bd0cbf0e-0d17-4cc7-99b2-cc5526cb0ea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11a233b00bfb4a10acc46f21156d1baa', 'request_id': 'req-bd0cbf0e-0d17-4cc7-99b2-cc5526cb0ea1'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:43.738748 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-bd0cbf0e-0d17-4cc7-99b2-cc5526cb0ea1 admin admin] The token cache is being invalidate because project 11a233b00bfb4a10acc46f21156d1baa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:12:43.739071 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-bd0cbf0e-0d17-4cc7-99b2-cc5526cb0ea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:43.746772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-bd0cbf0e-0d17-4cc7-99b2-cc5526cb0ea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:43.759144 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1038/2073] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:12:43 2022] DELETE /identity/v3/projects/11a233b00bfb4a10acc46f21156d1baa => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:43.761922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430ee30-aa46-4dd0-b5f4-feb69e93bcbe None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:43.766655 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430ee30-aa46-4dd0-b5f4-feb69e93bcbe None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:43.767070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430ee30-aa46-4dd0-b5f4-feb69e93bcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:43.790668 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-e430ee30-aa46-4dd0-b5f4-feb69e93bcbe None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:43.792368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e430ee30-aa46-4dd0-b5f4-feb69e93bcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:43.895705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1036/2074] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:43.905956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef5743-72f8-407f-8377-bab433352f46 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:43.906263 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef5743-72f8-407f-8377-bab433352f46 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:43.906545 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef5743-72f8-407f-8377-bab433352f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:43.943648 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-3aef5743-72f8-407f-8377-bab433352f46 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:43.945009 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3aef5743-72f8-407f-8377-bab433352f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:43.954434 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1039/2075] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:43.959194 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430ee30-aa46-4dd0-b5f4-feb69e93bcbe None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:43.970634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430ee30-aa46-4dd0-b5f4-feb69e93bcbe None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:43.975690 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189d683-db41-4d9f-8f91-bc9e38281fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:43.976342 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189d683-db41-4d9f-8f91-bc9e38281fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:43.976926 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189d683-db41-4d9f-8f91-bc9e38281fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:43.977188 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189d683-db41-4d9f-8f91-bc9e38281fa9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:43.977950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189d683-db41-4d9f-8f91-bc9e38281fa9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:12:43.978254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189d683-db41-4d9f-8f91-bc9e38281fa9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:43.980151 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189d683-db41-4d9f-8f91-bc9e38281fa9 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:43.995137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1037/2076] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:12:44.000812 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef5743-72f8-407f-8377-bab433352f46 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:44.016582 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef5743-72f8-407f-8377-bab433352f46 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:44.029222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793eae6-5032-4b8f-a8e0-c7e694c56ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:44.029222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793eae6-5032-4b8f-a8e0-c7e694c56ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:44.029222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793eae6-5032-4b8f-a8e0-c7e694c56ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:44.029222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793eae6-5032-4b8f-a8e0-c7e694c56ea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:44.029774 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793eae6-5032-4b8f-a8e0-c7e694c56ea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:44.029774 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793eae6-5032-4b8f-a8e0-c7e694c56ea6 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:44.269708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1040/2077] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:12:43 2022] POST /identity/v3/projects => generated 365 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:44.274294 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189d683-db41-4d9f-8f91-bc9e38281fa9 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:44.301198 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189d683-db41-4d9f-8f91-bc9e38281fa9 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:44.427483 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d37a4-b998-4739-891c-b26ab1c7b7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:44.428726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d37a4-b998-4739-891c-b26ab1c7b7f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:44.429362 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d37a4-b998-4739-891c-b26ab1c7b7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:44.430988 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d37a4-b998-4739-891c-b26ab1c7b7f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:44.431870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d37a4-b998-4739-891c-b26ab1c7b7f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:44.439289 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d37a4-b998-4739-891c-b26ab1c7b7f0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:44.576174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1038/2078] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:12:44 2022] POST /identity/v3/users => generated 283 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:12:44.580746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793eae6-5032-4b8f-a8e0-c7e694c56ea6 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:44.596519 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793eae6-5032-4b8f-a8e0-c7e694c56ea6 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:44.619392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f4638-e537-4ba7-a130-a83a8abfc689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:44.624680 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f4638-e537-4ba7-a130-a83a8abfc689 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:44.624680 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f4638-e537-4ba7-a130-a83a8abfc689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:44.624680 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f4638-e537-4ba7-a130-a83a8abfc689 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:44.624680 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f4638-e537-4ba7-a130-a83a8abfc689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:44.624680 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f4638-e537-4ba7-a130-a83a8abfc689 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:44.637677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1041/2079] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:12:44.642068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d37a4-b998-4739-891c-b26ab1c7b7f0 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:44.654945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d37a4-b998-4739-891c-b26ab1c7b7f0 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:44.661284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc83fd-2702-4949-b401-f30a478f1bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:44.662145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc83fd-2702-4949-b401-f30a478f1bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:44.662532 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc83fd-2702-4949-b401-f30a478f1bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:44.663153 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc83fd-2702-4949-b401-f30a478f1bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:44.664025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc83fd-2702-4949-b401-f30a478f1bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:44.665784 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc83fd-2702-4949-b401-f30a478f1bd4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:44.682668 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1039/2080] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:44.684515 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f4638-e537-4ba7-a130-a83a8abfc689 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:44.713032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f4638-e537-4ba7-a130-a83a8abfc689 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:44.715631 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc83fd-2702-4949-b401-f30a478f1bd4 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:44.720379 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-60cc83fd-2702-4949-b401-f30a478f1bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:44.734281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f94bf-abf6-4bdd-8516-cc6e34f4d5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:44.735644 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f94bf-abf6-4bdd-8516-cc6e34f4d5bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:44.737752 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f94bf-abf6-4bdd-8516-cc6e34f4d5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:44.738297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f94bf-abf6-4bdd-8516-cc6e34f4d5bc admin admin] PATH_INFO: `/v3/projects/5982b82cd0eb442aaf4fad9a89e015d4/users/b9b9c3168d844fdc9f57475fdcca34ac/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:44.763942 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc83fd-2702-4949-b401-f30a478f1bd4 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:44.772999 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165db677-91f6-4c58-b4bd-c7a71bae70af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:44.773906 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165db677-91f6-4c58-b4bd-c7a71bae70af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:44.774376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165db677-91f6-4c58-b4bd-c7a71bae70af admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:44.774664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165db677-91f6-4c58-b4bd-c7a71bae70af admin admin] PATH_INFO: `/v3/projects/809727ae4d354ff08d7f20f8f0c133d3` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:44.776380 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165db677-91f6-4c58-b4bd-c7a71bae70af admin admin] RBAC: Authorizing `identity:delete_project(project_id=809727ae4d354ff08d7f20f8f0c133d3)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:44.780093 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165db677-91f6-4c58-b4bd-c7a71bae70af admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:44.787746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f94bf-abf6-4bdd-8516-cc6e34f4d5bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5982b82cd0eb442aaf4fad9a89e015d4, user_id=b9b9c3168d844fdc9f57475fdcca34ac, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:44.802689 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f94bf-abf6-4bdd-8516-cc6e34f4d5bc admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:44.991027 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1042/2081] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:44 2022] PUT /identity/v3/projects/5982b82cd0eb442aaf4fad9a89e015d4/users/b9b9c3168d844fdc9f57475fdcca34ac/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:45.021986 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-165db677-91f6-4c58-b4bd-c7a71bae70af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '809727ae4d354ff08d7f20f8f0c133d3', 'request_id': 'req-165db677-91f6-4c58-b4bd-c7a71bae70af'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:45.025272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d282a-9467-4ef5-918b-61cb2cc8296f None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:45.025272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d282a-9467-4ef5-918b-61cb2cc8296f None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:45.025272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d282a-9467-4ef5-918b-61cb2cc8296f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:45.035346 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-165db677-91f6-4c58-b4bd-c7a71bae70af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '809727ae4d354ff08d7f20f8f0c133d3', 'request_id': 'req-165db677-91f6-4c58-b4bd-c7a71bae70af'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:45.066622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-165db677-91f6-4c58-b4bd-c7a71bae70af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '809727ae4d354ff08d7f20f8f0c133d3', 'request_id': 'req-165db677-91f6-4c58-b4bd-c7a71bae70af'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:45.124635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-7f6d282a-9467-4ef5-918b-61cb2cc8296f None None] MFA Rules not processed for user `b9b9c3168d844fdc9f57475fdcca34ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:45.127017 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7f6d282a-9467-4ef5-918b-61cb2cc8296f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:45.196544 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-165db677-91f6-4c58-b4bd-c7a71bae70af admin admin] The token cache is being invalidate because project 809727ae4d354ff08d7f20f8f0c133d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:12:45.197187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-165db677-91f6-4c58-b4bd-c7a71bae70af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:45.198095 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-165db677-91f6-4c58-b4bd-c7a71bae70af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:45.200154 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1040/2082] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:12:44 2022] DELETE /identity/v3/projects/809727ae4d354ff08d7f20f8f0c133d3 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:12:45.208801 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed30bd-d599-4411-bfc5-6b5649b2b866 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:45.209778 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed30bd-d599-4411-bfc5-6b5649b2b866 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:45.210514 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed30bd-d599-4411-bfc5-6b5649b2b866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:45.263416 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3fed30bd-d599-4411-bfc5-6b5649b2b866 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:45.265712 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3fed30bd-d599-4411-bfc5-6b5649b2b866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:45.275003 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1043/2083] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:45.294037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d282a-9467-4ef5-918b-61cb2cc8296f None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:45.298070 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7f6d282a-9467-4ef5-918b-61cb2cc8296f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:45.432735 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1041/2084] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:45.455085 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4d293-b17f-41ef-a36e-37cf6a478d3a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:45.455443 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4d293-b17f-41ef-a36e-37cf6a478d3a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:45.455667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4d293-b17f-41ef-a36e-37cf6a478d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:45.477832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d282a-9467-4ef5-918b-61cb2cc8296f None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:45.481886 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b6432-fc4b-48c2-ab0e-244c05f908ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:45.482635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b6432-fc4b-48c2-ab0e-244c05f908ff service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:45.483976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b6432-fc4b-48c2-ab0e-244c05f908ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:45.484307 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b6432-fc4b-48c2-ab0e-244c05f908ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:45.485789 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-28d4d293-b17f-41ef-a36e-37cf6a478d3a None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:45.485888 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-28d4d293-b17f-41ef-a36e-37cf6a478d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:45.494934 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1042/2085] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:45.496497 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b6432-fc4b-48c2-ab0e-244c05f908ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:45.498255 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b6432-fc4b-48c2-ab0e-244c05f908ff service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:45.500425 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4d293-b17f-41ef-a36e-37cf6a478d3a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:45.510588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4d293-b17f-41ef-a36e-37cf6a478d3a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:45.510788 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1044/2086] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:45 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:12:45.516028 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca22a0-fc09-46de-bffc-b86a01760d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:45.516728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca22a0-fc09-46de-bffc-b86a01760d95 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:45.517013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca22a0-fc09-46de-bffc-b86a01760d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:45.517290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca22a0-fc09-46de-bffc-b86a01760d95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:45.518041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca22a0-fc09-46de-bffc-b86a01760d95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:12:45.518396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca22a0-fc09-46de-bffc-b86a01760d95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:45.520090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca22a0-fc09-46de-bffc-b86a01760d95 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:45.529484 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:45.535778 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1043/2087] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:12:45.539813 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b6432-fc4b-48c2-ab0e-244c05f908ff service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:45.553492 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b6432-fc4b-48c2-ab0e-244c05f908ff service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:45.559147 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63458fc-a2e6-4feb-94dc-3f29e0a9c907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:45.559958 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63458fc-a2e6-4feb-94dc-3f29e0a9c907 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:45.560306 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63458fc-a2e6-4feb-94dc-3f29e0a9c907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:45.560651 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63458fc-a2e6-4feb-94dc-3f29e0a9c907 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:45.562063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63458fc-a2e6-4feb-94dc-3f29e0a9c907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:45.564395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63458fc-a2e6-4feb-94dc-3f29e0a9c907 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:45.706621 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1045/2088] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:12:45 2022] POST /identity/v3/projects => generated 369 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:12:45.711562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca22a0-fc09-46de-bffc-b86a01760d95 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:45.734646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca22a0-fc09-46de-bffc-b86a01760d95 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:45.749254 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:45.892519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf416a-b0b2-456e-a3c9-e7a29b2b2fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:45.893270 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf416a-b0b2-456e-a3c9-e7a29b2b2fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:45.893550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf416a-b0b2-456e-a3c9-e7a29b2b2fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:45.893967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf416a-b0b2-456e-a3c9-e7a29b2b2fb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:45.894521 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf416a-b0b2-456e-a3c9-e7a29b2b2fb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:45.896800 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf416a-b0b2-456e-a3c9-e7a29b2b2fb3 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:46.049865 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1044/2089] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:12:45 2022] POST /identity/v3/users => generated 285 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:12:46.051251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63458fc-a2e6-4feb-94dc-3f29e0a9c907 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:46.066783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63458fc-a2e6-4feb-94dc-3f29e0a9c907 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:46.091987 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613096aa-b0be-4aa4-9101-28937cd8b7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:46.093733 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613096aa-b0be-4aa4-9101-28937cd8b7e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:46.094176 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613096aa-b0be-4aa4-9101-28937cd8b7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:46.094592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613096aa-b0be-4aa4-9101-28937cd8b7e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:46.095926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613096aa-b0be-4aa4-9101-28937cd8b7e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:46.098880 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613096aa-b0be-4aa4-9101-28937cd8b7e0 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:46.131079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1046/2090] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:46.136731 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf416a-b0b2-456e-a3c9-e7a29b2b2fb3 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:46.150327 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf416a-b0b2-456e-a3c9-e7a29b2b2fb3 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:46.156240 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f397904-07cf-47ca-92db-a96486870757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:46.157219 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f397904-07cf-47ca-92db-a96486870757 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:46.157706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f397904-07cf-47ca-92db-a96486870757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:46.158135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f397904-07cf-47ca-92db-a96486870757 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:46.159057 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f397904-07cf-47ca-92db-a96486870757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:46.161135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f397904-07cf-47ca-92db-a96486870757 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:46.174930 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1045/2091] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:46.179059 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613096aa-b0be-4aa4-9101-28937cd8b7e0 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:46.198128 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613096aa-b0be-4aa4-9101-28937cd8b7e0 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:46.206521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01ed04-70ef-4e31-a8de-acea4c190586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:46.208182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01ed04-70ef-4e31-a8de-acea4c190586 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:46.208646 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01ed04-70ef-4e31-a8de-acea4c190586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:46.209127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01ed04-70ef-4e31-a8de-acea4c190586 admin admin] PATH_INFO: `/v3/projects/d771120895ed4423a2dbf25c1c9b2732/users/ff8ae9f8a9db47fdbc5e63fffb2f78b1/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:46.237499 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01ed04-70ef-4e31-a8de-acea4c190586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d771120895ed4423a2dbf25c1c9b2732, user_id=ff8ae9f8a9db47fdbc5e63fffb2f78b1, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:46.242837 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01ed04-70ef-4e31-a8de-acea4c190586 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:46.393758 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1047/2092] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:46 2022] PUT /identity/v3/projects/d771120895ed4423a2dbf25c1c9b2732/users/ff8ae9f8a9db47fdbc5e63fffb2f78b1/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:46.406246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc48f2-6bdf-4484-b35a-167c56c8bcbb None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:46.406790 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc48f2-6bdf-4484-b35a-167c56c8bcbb None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:46.407612 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc48f2-6bdf-4484-b35a-167c56c8bcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:46.447532 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-34cc48f2-6bdf-4484-b35a-167c56c8bcbb None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:46.449709 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-34cc48f2-6bdf-4484-b35a-167c56c8bcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:46.527981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:46.551040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1046/2093] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:12:46.560451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01ed04-70ef-4e31-a8de-acea4c190586 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:46.565009 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-8f01ed04-70ef-4e31-a8de-acea4c190586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:46.674146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01ed04-70ef-4e31-a8de-acea4c190586 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:46.678048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a49fee-2553-4323-a471-6e070fbf62b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:46.678985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a49fee-2553-4323-a471-6e070fbf62b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:46.679170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a49fee-2553-4323-a471-6e070fbf62b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:46.679369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a49fee-2553-4323-a471-6e070fbf62b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:46.692406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a49fee-2553-4323-a471-6e070fbf62b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:46.694121 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a49fee-2553-4323-a471-6e070fbf62b4 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:46.706531 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1048/2094] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:49.957576 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc48f2-6bdf-4484-b35a-167c56c8bcbb None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:49.960555 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-34cc48f2-6bdf-4484-b35a-167c56c8bcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:50.010567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc48f2-6bdf-4484-b35a-167c56c8bcbb None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:50.027834 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a078a4-9d9a-48ab-814a-44864825ef17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:50.028740 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a078a4-9d9a-48ab-814a-44864825ef17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:50.029022 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a078a4-9d9a-48ab-814a-44864825ef17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:50.029296 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a078a4-9d9a-48ab-814a-44864825ef17 admin admin] PATH_INFO: `/v3/users/b7a1047afa4940fa969e0ca5fc19dbb8` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:50.032351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a078a4-9d9a-48ab-814a-44864825ef17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7a1047afa4940fa969e0ca5fc19dbb8)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:50.034579 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a078a4-9d9a-48ab-814a-44864825ef17 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:50.456744 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-88a078a4-9d9a-48ab-814a-44864825ef17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7a1047afa4940fa969e0ca5fc19dbb8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:50.555102 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-88a078a4-9d9a-48ab-814a-44864825ef17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7a1047afa4940fa969e0ca5fc19dbb8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:50.556153 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-88a078a4-9d9a-48ab-814a-44864825ef17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7a1047afa4940fa969e0ca5fc19dbb8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:50.558472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-88a078a4-9d9a-48ab-814a-44864825ef17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7a1047afa4940fa969e0ca5fc19dbb8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:50.575213 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-88a078a4-9d9a-48ab-814a-44864825ef17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7a1047afa4940fa969e0ca5fc19dbb8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:50.634984 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1047/2095] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:12:49 2022] DELETE /identity/v3/users/b7a1047afa4940fa969e0ca5fc19dbb8 => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:50.641319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a612b86-3386-41da-8cd1-87cd62d0e629 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:50.642094 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a612b86-3386-41da-8cd1-87cd62d0e629 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:50.642094 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a612b86-3386-41da-8cd1-87cd62d0e629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:50.679712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-4a612b86-3386-41da-8cd1-87cd62d0e629 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:50.681861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-4a612b86-3386-41da-8cd1-87cd62d0e629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:50.765069 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:50.796100 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1049/2096] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:50.808306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a078a4-9d9a-48ab-814a-44864825ef17 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:50.816063 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-88a078a4-9d9a-48ab-814a-44864825ef17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:50.842581 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:50.932579 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a078a4-9d9a-48ab-814a-44864825ef17 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:50.938300 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214faa04-ff1b-431a-9446-ea5497ce5e45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:50.939254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214faa04-ff1b-431a-9446-ea5497ce5e45 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:50.939411 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214faa04-ff1b-431a-9446-ea5497ce5e45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:50.939710 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214faa04-ff1b-431a-9446-ea5497ce5e45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:50.956687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214faa04-ff1b-431a-9446-ea5497ce5e45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:50.958266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214faa04-ff1b-431a-9446-ea5497ce5e45 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:50.986913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1048/2097] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:51.939232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a612b86-3386-41da-8cd1-87cd62d0e629 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:51.941824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-4a612b86-3386-41da-8cd1-87cd62d0e629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:51.958625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a612b86-3386-41da-8cd1-87cd62d0e629 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:51.964880 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f12b7-2a6f-48f5-82ce-9bc3bf6e504c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:51.966246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f12b7-2a6f-48f5-82ce-9bc3bf6e504c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:51.966667 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f12b7-2a6f-48f5-82ce-9bc3bf6e504c admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:51.966781 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f12b7-2a6f-48f5-82ce-9bc3bf6e504c admin admin] PATH_INFO: `/v3/projects/7247dd612f6d480485a5857593c36831` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:51.969190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f12b7-2a6f-48f5-82ce-9bc3bf6e504c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7247dd612f6d480485a5857593c36831)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:51.971845 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f12b7-2a6f-48f5-82ce-9bc3bf6e504c admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:52.108374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-731f12b7-2a6f-48f5-82ce-9bc3bf6e504c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7247dd612f6d480485a5857593c36831', 'request_id': 'req-731f12b7-2a6f-48f5-82ce-9bc3bf6e504c'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:52.116543 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-731f12b7-2a6f-48f5-82ce-9bc3bf6e504c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7247dd612f6d480485a5857593c36831', 'request_id': 'req-731f12b7-2a6f-48f5-82ce-9bc3bf6e504c'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:52.125836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-731f12b7-2a6f-48f5-82ce-9bc3bf6e504c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7247dd612f6d480485a5857593c36831', 'request_id': 'req-731f12b7-2a6f-48f5-82ce-9bc3bf6e504c'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:52.285634 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-731f12b7-2a6f-48f5-82ce-9bc3bf6e504c admin admin] The token cache is being invalidate because project 7247dd612f6d480485a5857593c36831 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:12:52.285850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-731f12b7-2a6f-48f5-82ce-9bc3bf6e504c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:52.286624 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-731f12b7-2a6f-48f5-82ce-9bc3bf6e504c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:52.288706 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1050/2098] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:12:51 2022] DELETE /identity/v3/projects/7247dd612f6d480485a5857593c36831 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:52.299051 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214faa04-ff1b-431a-9446-ea5497ce5e45 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:52.337767 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214faa04-ff1b-431a-9446-ea5497ce5e45 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:52.436951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1fd72-f491-4ef3-a2b1-971b5e510966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:52.437840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1fd72-f491-4ef3-a2b1-971b5e510966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:52.438452 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1fd72-f491-4ef3-a2b1-971b5e510966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:52.438731 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1fd72-f491-4ef3-a2b1-971b5e510966 admin admin] PATH_INFO: `/v3/users/6c72d8fe4dfd40cfa26c70662d256019` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:52.440577 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1fd72-f491-4ef3-a2b1-971b5e510966 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c72d8fe4dfd40cfa26c70662d256019)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:52.443053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1fd72-f491-4ef3-a2b1-971b5e510966 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:52.726930 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-82e1fd72-f491-4ef3-a2b1-971b5e510966 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c72d8fe4dfd40cfa26c70662d256019'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:52.849470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-82e1fd72-f491-4ef3-a2b1-971b5e510966 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c72d8fe4dfd40cfa26c70662d256019'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:52.856222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-82e1fd72-f491-4ef3-a2b1-971b5e510966 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c72d8fe4dfd40cfa26c70662d256019'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:52.859391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-82e1fd72-f491-4ef3-a2b1-971b5e510966 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c72d8fe4dfd40cfa26c70662d256019'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:52.870512 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-82e1fd72-f491-4ef3-a2b1-971b5e510966 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c72d8fe4dfd40cfa26c70662d256019'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:52.908748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1049/2099] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:12:52 2022] DELETE /identity/v3/users/6c72d8fe4dfd40cfa26c70662d256019 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:53.809233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f12b7-2a6f-48f5-82ce-9bc3bf6e504c admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:53.812406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-731f12b7-2a6f-48f5-82ce-9bc3bf6e504c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:53.841110 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f12b7-2a6f-48f5-82ce-9bc3bf6e504c admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:53.954862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf02beb2-70b5-4052-838c-a0f1b9641b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:53.955796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02beb2-70b5-4052-838c-a0f1b9641b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:53.956059 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02beb2-70b5-4052-838c-a0f1b9641b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:53.956327 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02beb2-70b5-4052-838c-a0f1b9641b61 admin admin] PATH_INFO: `/v3/projects/d904b5a36600462abbc6e7d4b8f69d8b` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:53.957573 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf02beb2-70b5-4052-838c-a0f1b9641b61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d904b5a36600462abbc6e7d4b8f69d8b)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:53.959595 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf02beb2-70b5-4052-838c-a0f1b9641b61 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:54.114395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52e16d-508e-406f-b8bb-e37001c7663b None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:54.118932 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52e16d-508e-406f-b8bb-e37001c7663b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:54.118932 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52e16d-508e-406f-b8bb-e37001c7663b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:54.153919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-9c52e16d-508e-406f-b8bb-e37001c7663b None None] MFA Rules not processed for user `8ba8498f6fbc40c18cadbb4ce6667855`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:54.156129 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9c52e16d-508e-406f-b8bb-e37001c7663b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:54.164124 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-cf02beb2-70b5-4052-838c-a0f1b9641b61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd904b5a36600462abbc6e7d4b8f69d8b', 'request_id': 'req-cf02beb2-70b5-4052-838c-a0f1b9641b61'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:54.180329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-cf02beb2-70b5-4052-838c-a0f1b9641b61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd904b5a36600462abbc6e7d4b8f69d8b', 'request_id': 'req-cf02beb2-70b5-4052-838c-a0f1b9641b61'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:54.197380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-cf02beb2-70b5-4052-838c-a0f1b9641b61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd904b5a36600462abbc6e7d4b8f69d8b', 'request_id': 'req-cf02beb2-70b5-4052-838c-a0f1b9641b61'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:54.206177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:54.258951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1050/2100] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:54.294515 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-cf02beb2-70b5-4052-838c-a0f1b9641b61 admin admin] The token cache is being invalidate because project d904b5a36600462abbc6e7d4b8f69d8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:12:54.294817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-cf02beb2-70b5-4052-838c-a0f1b9641b61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:54.296675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-cf02beb2-70b5-4052-838c-a0f1b9641b61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:54.299319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1051/2101] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:12:53 2022] DELETE /identity/v3/projects/d904b5a36600462abbc6e7d4b8f69d8b => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:12:54.305477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb9975-e969-40a9-bcd3-02bccf2ae1c0 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:54.306189 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb9975-e969-40a9-bcd3-02bccf2ae1c0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:54.307515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb9975-e969-40a9-bcd3-02bccf2ae1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:54.338821 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-eebb9975-e969-40a9-bcd3-02bccf2ae1c0 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:54.340883 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-eebb9975-e969-40a9-bcd3-02bccf2ae1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:54.490041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1051/2102] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:54.500217 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236446a-b1c8-4b26-820e-c88b1bbdbeb6 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:54.500506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236446a-b1c8-4b26-820e-c88b1bbdbeb6 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:54.500791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236446a-b1c8-4b26-820e-c88b1bbdbeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:54.553051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-b236446a-b1c8-4b26-820e-c88b1bbdbeb6 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:54.554284 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b236446a-b1c8-4b26-820e-c88b1bbdbeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:54.594358 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1052/2103] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:12:54.598923 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb9975-e969-40a9-bcd3-02bccf2ae1c0 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:54.617543 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb9975-e969-40a9-bcd3-02bccf2ae1c0 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:54.628588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a5fb3-4bb7-4438-a3bc-386a562306ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:54.633743 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a5fb3-4bb7-4438-a3bc-386a562306ca admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:54.633743 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a5fb3-4bb7-4438-a3bc-386a562306ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:54.633743 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a5fb3-4bb7-4438-a3bc-386a562306ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:54.633743 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a5fb3-4bb7-4438-a3bc-386a562306ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:12:54.633743 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a5fb3-4bb7-4438-a3bc-386a562306ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:54.633743 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a5fb3-4bb7-4438-a3bc-386a562306ca admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:54.662730 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:54.667284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1052/2104] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:12:54.674036 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236446a-b1c8-4b26-820e-c88b1bbdbeb6 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:54.686503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236446a-b1c8-4b26-820e-c88b1bbdbeb6 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:54.699436 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae179ca0-b154-4ac4-b6ca-94d99f073121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:54.701001 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae179ca0-b154-4ac4-b6ca-94d99f073121 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:54.702581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae179ca0-b154-4ac4-b6ca-94d99f073121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:54.703368 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae179ca0-b154-4ac4-b6ca-94d99f073121 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:54.704953 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae179ca0-b154-4ac4-b6ca-94d99f073121 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:54.709626 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae179ca0-b154-4ac4-b6ca-94d99f073121 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:54.985600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1053/2105] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:12:54 2022] POST /identity/v3/projects => generated 371 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:54.988966 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a5fb3-4bb7-4438-a3bc-386a562306ca admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:55.019947 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a5fb3-4bb7-4438-a3bc-386a562306ca admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:55.127287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a8f9c-b041-4906-83c4-3782bae4c655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:55.128824 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a8f9c-b041-4906-83c4-3782bae4c655 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:55.129330 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a8f9c-b041-4906-83c4-3782bae4c655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:55.143171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a8f9c-b041-4906-83c4-3782bae4c655 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:55.144524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a8f9c-b041-4906-83c4-3782bae4c655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:55.151080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a8f9c-b041-4906-83c4-3782bae4c655 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:55.291396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1053/2106] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:12:54 2022] POST /identity/v3/users => generated 286 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:12:55.296871 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae179ca0-b154-4ac4-b6ca-94d99f073121 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:55.313292 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae179ca0-b154-4ac4-b6ca-94d99f073121 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:55.352366 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbd2ac-0571-4052-9465-06604b47a526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:55.357035 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbd2ac-0571-4052-9465-06604b47a526 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:55.360052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbd2ac-0571-4052-9465-06604b47a526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:55.360232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbd2ac-0571-4052-9465-06604b47a526 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:55.362877 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fbd2ac-0571-4052-9465-06604b47a526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:55.370696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fbd2ac-0571-4052-9465-06604b47a526 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:55.429837 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1054/2107] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:55.433637 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a8f9c-b041-4906-83c4-3782bae4c655 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:55.469116 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a8f9c-b041-4906-83c4-3782bae4c655 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:55.491502 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b23445-4b06-48a3-a73d-68a8e883e1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:55.492205 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b23445-4b06-48a3-a73d-68a8e883e1c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:55.492497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b23445-4b06-48a3-a73d-68a8e883e1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:55.492770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b23445-4b06-48a3-a73d-68a8e883e1c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:55.493621 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b23445-4b06-48a3-a73d-68a8e883e1c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:55.497030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b23445-4b06-48a3-a73d-68a8e883e1c0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:55.553243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1054/2108] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:55.558951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbd2ac-0571-4052-9465-06604b47a526 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:55.605918 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbd2ac-0571-4052-9465-06604b47a526 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:55.613384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1874e0e-8760-4acf-bcc1-9183fbab75fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:55.614488 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1874e0e-8760-4acf-bcc1-9183fbab75fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:55.614990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1874e0e-8760-4acf-bcc1-9183fbab75fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:55.615493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1874e0e-8760-4acf-bcc1-9183fbab75fb admin admin] PATH_INFO: `/v3/projects/6eef99bec3d241679bd53c1152d14b1a/users/d37bb98b06634afabf3dfab6dc47ff2a/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:55.653295 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1874e0e-8760-4acf-bcc1-9183fbab75fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eef99bec3d241679bd53c1152d14b1a, user_id=d37bb98b06634afabf3dfab6dc47ff2a, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:55.658615 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1874e0e-8760-4acf-bcc1-9183fbab75fb admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:55.837408 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d98e17-5d9c-4926-979d-8d9cc3ec6a8b None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:55.837971 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d98e17-5d9c-4926-979d-8d9cc3ec6a8b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:55.838865 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d98e17-5d9c-4926-979d-8d9cc3ec6a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:55.866783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1055/2109] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:55 2022] PUT /identity/v3/projects/6eef99bec3d241679bd53c1152d14b1a/users/d37bb98b06634afabf3dfab6dc47ff2a/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:55.942982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d2d98e17-5d9c-4926-979d-8d9cc3ec6a8b None None] MFA Rules not processed for user `d37bb98b06634afabf3dfab6dc47ff2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:55.944638 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d2d98e17-5d9c-4926-979d-8d9cc3ec6a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:56.142574 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1055/2110] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:56.162825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1874e0e-8760-4acf-bcc1-9183fbab75fb admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:56.167613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b1874e0e-8760-4acf-bcc1-9183fbab75fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:56.448806 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1874e0e-8760-4acf-bcc1-9183fbab75fb admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:56.455044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc46f30-36a9-4bb0-8d87-b780cd75270d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:56.457811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc46f30-36a9-4bb0-8d87-b780cd75270d service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:56.469230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc46f30-36a9-4bb0-8d87-b780cd75270d service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:56.469230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc46f30-36a9-4bb0-8d87-b780cd75270d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:56.520152 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc46f30-36a9-4bb0-8d87-b780cd75270d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:56.524424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc46f30-36a9-4bb0-8d87-b780cd75270d service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:56.553756 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1056/2111] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:12:56.583880 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d98e17-5d9c-4926-979d-8d9cc3ec6a8b None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:56.810110 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d98e17-5d9c-4926-979d-8d9cc3ec6a8b None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:56.810110 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b4f4a-8eca-48a7-b791-be1bb9e988d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:56.810110 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b4f4a-8eca-48a7-b791-be1bb9e988d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:56.810110 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b4f4a-8eca-48a7-b791-be1bb9e988d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:56.811695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b4f4a-8eca-48a7-b791-be1bb9e988d9 admin admin] PATH_INFO: `/v3/users/d37bb98b06634afabf3dfab6dc47ff2a` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:56.811695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b4f4a-8eca-48a7-b791-be1bb9e988d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d37bb98b06634afabf3dfab6dc47ff2a)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:56.811695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b4f4a-8eca-48a7-b791-be1bb9e988d9 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:56.814020 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34668bde-10e1-4c5d-b0bb-f646672517ab None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:56.818974 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34668bde-10e1-4c5d-b0bb-f646672517ab None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:56.818974 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34668bde-10e1-4c5d-b0bb-f646672517ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:56.868089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-34668bde-10e1-4c5d-b0bb-f646672517ab None None] MFA Rules not processed for user `ff8ae9f8a9db47fdbc5e63fffb2f78b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:56.869889 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-34668bde-10e1-4c5d-b0bb-f646672517ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:57.063414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1057/2112] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:56 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:57.066421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34668bde-10e1-4c5d-b0bb-f646672517ab None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:57.076657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-34668bde-10e1-4c5d-b0bb-f646672517ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:57.095368 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34668bde-10e1-4c5d-b0bb-f646672517ab None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:57.101132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04c58e-aeb4-41c2-8fbb-b47532522b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:57.101906 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04c58e-aeb4-41c2-8fbb-b47532522b8b admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:57.103058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04c58e-aeb4-41c2-8fbb-b47532522b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:57.103372 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04c58e-aeb4-41c2-8fbb-b47532522b8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:57.104373 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04c58e-aeb4-41c2-8fbb-b47532522b8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:57.106658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04c58e-aeb4-41c2-8fbb-b47532522b8b admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:57.208902 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2e1b4f4a-8eca-48a7-b791-be1bb9e988d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd37bb98b06634afabf3dfab6dc47ff2a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:57.262796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1058/2113] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:12:57 2022] POST /identity/v3/projects => generated 373 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:57.266806 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04c58e-aeb4-41c2-8fbb-b47532522b8b admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:57.279676 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2e1b4f4a-8eca-48a7-b791-be1bb9e988d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd37bb98b06634afabf3dfab6dc47ff2a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:57.280670 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2e1b4f4a-8eca-48a7-b791-be1bb9e988d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd37bb98b06634afabf3dfab6dc47ff2a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:57.281579 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2e1b4f4a-8eca-48a7-b791-be1bb9e988d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd37bb98b06634afabf3dfab6dc47ff2a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:57.291766 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2e1b4f4a-8eca-48a7-b791-be1bb9e988d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd37bb98b06634afabf3dfab6dc47ff2a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:57.327481 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1056/2114] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:12:56 2022] DELETE /identity/v3/users/d37bb98b06634afabf3dfab6dc47ff2a => generated 0 bytes in 743 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:57.333008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295ecb9-6dd6-4d9e-9e96-856cc12e77ad None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:57.333648 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295ecb9-6dd6-4d9e-9e96-856cc12e77ad None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:57.334202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295ecb9-6dd6-4d9e-9e96-856cc12e77ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:57.385480 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5295ecb9-6dd6-4d9e-9e96-856cc12e77ad None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:57.392873 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5295ecb9-6dd6-4d9e-9e96-856cc12e77ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:57.432576 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04c58e-aeb4-41c2-8fbb-b47532522b8b admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:57.494978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1057/2115] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:12:57.508535 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295ecb9-6dd6-4d9e-9e96-856cc12e77ad None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:57.511042 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5295ecb9-6dd6-4d9e-9e96-856cc12e77ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:57.532892 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248fa4f-e70a-44af-90fc-29ee79c7960e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:57.533601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248fa4f-e70a-44af-90fc-29ee79c7960e admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:57.533888 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248fa4f-e70a-44af-90fc-29ee79c7960e admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:57.534168 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248fa4f-e70a-44af-90fc-29ee79c7960e admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:57.534917 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1248fa4f-e70a-44af-90fc-29ee79c7960e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:57.537617 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1248fa4f-e70a-44af-90fc-29ee79c7960e admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:57.648857 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1059/2116] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:12:57 2022] POST /identity/v3/users => generated 293 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:12:57.660256 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248fa4f-e70a-44af-90fc-29ee79c7960e admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:57.662989 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-1248fa4f-e70a-44af-90fc-29ee79c7960e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:57.689368 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248fa4f-e70a-44af-90fc-29ee79c7960e admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:57.690814 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295ecb9-6dd6-4d9e-9e96-856cc12e77ad None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:57.696057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9953390-921f-4abc-937f-424518a44ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:57.696882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9953390-921f-4abc-937f-424518a44ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:57.697177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9953390-921f-4abc-937f-424518a44ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:57.697547 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9953390-921f-4abc-937f-424518a44ae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:57.698353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9953390-921f-4abc-937f-424518a44ae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:57.700790 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9953390-921f-4abc-937f-424518a44ae7 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:57.704635 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b42523-706d-46cb-948d-3f943bec93bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:57.705955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b42523-706d-46cb-948d-3f943bec93bd service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:57.706558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b42523-706d-46cb-948d-3f943bec93bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:57.707668 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b42523-706d-46cb-948d-3f943bec93bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:57.727039 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1060/2117] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:57.728024 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9953390-921f-4abc-937f-424518a44ae7 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:57.734465 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b42523-706d-46cb-948d-3f943bec93bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:57.737017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b42523-706d-46cb-948d-3f943bec93bd service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:57.741229 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9953390-921f-4abc-937f-424518a44ae7 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:57.753010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd116a9d-ef5f-4e7f-a88d-e19f417f1600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:57.753926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd116a9d-ef5f-4e7f-a88d-e19f417f1600 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:57.754214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd116a9d-ef5f-4e7f-a88d-e19f417f1600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:57.754488 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd116a9d-ef5f-4e7f-a88d-e19f417f1600 admin admin] PATH_INFO: `/v3/projects/9d3ee05d7001436a83c3dc5a7a2c3628/users/e41aae123ec64a12a614635f23368cd2/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:57.768616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1058/2118] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:12:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:12:57.803028 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd116a9d-ef5f-4e7f-a88d-e19f417f1600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d3ee05d7001436a83c3dc5a7a2c3628, user_id=e41aae123ec64a12a614635f23368cd2, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:57.806333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd116a9d-ef5f-4e7f-a88d-e19f417f1600 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:58.048357 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1061/2119] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:57 2022] PUT /identity/v3/projects/9d3ee05d7001436a83c3dc5a7a2c3628/users/e41aae123ec64a12a614635f23368cd2/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:58.053536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b42523-706d-46cb-948d-3f943bec93bd service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:58.081593 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b42523-706d-46cb-948d-3f943bec93bd service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:58.221933 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563158b-889a-4da9-ae41-a3af295d0eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:58.223446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563158b-889a-4da9-ae41-a3af295d0eaa admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:58.223795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563158b-889a-4da9-ae41-a3af295d0eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:58.224080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563158b-889a-4da9-ae41-a3af295d0eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:58.224918 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:58.225046 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563158b-889a-4da9-ae41-a3af295d0eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:58.226533 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563158b-889a-4da9-ae41-a3af295d0eaa admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:58.241814 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1059/2120] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:12:58.243889 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd116a9d-ef5f-4e7f-a88d-e19f417f1600 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:58.296095 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd116a9d-ef5f-4e7f-a88d-e19f417f1600 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:58.334846 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d97c2a-e8d1-4217-b482-ec00db81d814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:58.336531 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d97c2a-e8d1-4217-b482-ec00db81d814 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:58.342750 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d97c2a-e8d1-4217-b482-ec00db81d814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:58.343310 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d97c2a-e8d1-4217-b482-ec00db81d814 admin admin] PATH_INFO: `/v3/projects/9d3ee05d7001436a83c3dc5a7a2c3628/users/e41aae123ec64a12a614635f23368cd2/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:58.359518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d97c2a-e8d1-4217-b482-ec00db81d814 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d3ee05d7001436a83c3dc5a7a2c3628, user_id=e41aae123ec64a12a614635f23368cd2, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:58.372860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d97c2a-e8d1-4217-b482-ec00db81d814 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:58.435038 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1062/2121] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:12:58 2022] PUT /identity/v3/projects/9d3ee05d7001436a83c3dc5a7a2c3628/users/e41aae123ec64a12a614635f23368cd2/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:12:58.554194 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:12:59.049499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563158b-889a-4da9-ae41-a3af295d0eaa admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:59.051323 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f563158b-889a-4da9-ae41-a3af295d0eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:59.067847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563158b-889a-4da9-ae41-a3af295d0eaa admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:59.187056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0915a-f858-4188-8b03-967ed1e0dcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:59.191439 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0915a-f858-4188-8b03-967ed1e0dcb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:59.191439 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0915a-f858-4188-8b03-967ed1e0dcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:59.191439 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0915a-f858-4188-8b03-967ed1e0dcb5 admin admin] PATH_INFO: `/v3/projects/6eef99bec3d241679bd53c1152d14b1a` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:59.192733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0915a-f858-4188-8b03-967ed1e0dcb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eef99bec3d241679bd53c1152d14b1a)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:59.195532 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0915a-f858-4188-8b03-967ed1e0dcb5 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:59.364423 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f3a0915a-f858-4188-8b03-967ed1e0dcb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eef99bec3d241679bd53c1152d14b1a', 'request_id': 'req-f3a0915a-f858-4188-8b03-967ed1e0dcb5'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:59.380292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f3a0915a-f858-4188-8b03-967ed1e0dcb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eef99bec3d241679bd53c1152d14b1a', 'request_id': 'req-f3a0915a-f858-4188-8b03-967ed1e0dcb5'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:59.398435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f3a0915a-f858-4188-8b03-967ed1e0dcb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eef99bec3d241679bd53c1152d14b1a', 'request_id': 'req-f3a0915a-f858-4188-8b03-967ed1e0dcb5'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:59.570050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f3a0915a-f858-4188-8b03-967ed1e0dcb5 admin admin] The token cache is being invalidate because project 6eef99bec3d241679bd53c1152d14b1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:12:59.570347 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f3a0915a-f858-4188-8b03-967ed1e0dcb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:59.571406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f3a0915a-f858-4188-8b03-967ed1e0dcb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:12:59.572651 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1060/2122] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:12:59 2022] DELETE /identity/v3/projects/6eef99bec3d241679bd53c1152d14b1a => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:12:59.579366 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53893502-05e8-4cf8-9439-c9a69d0fe17e None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:59.579893 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53893502-05e8-4cf8-9439-c9a69d0fe17e None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:59.580021 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53893502-05e8-4cf8-9439-c9a69d0fe17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:59.658490 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-53893502-05e8-4cf8-9439-c9a69d0fe17e None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:59.662050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-53893502-05e8-4cf8-9439-c9a69d0fe17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:59.800691 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1063/2123] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:59.808910 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb7444-4944-4ed6-b482-e08f5b7d1f44 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:59.809565 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb7444-4944-4ed6-b482-e08f5b7d1f44 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:59.810018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb7444-4944-4ed6-b482-e08f5b7d1f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:59.833787 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b0bb7444-4944-4ed6-b482-e08f5b7d1f44 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:12:59.836014 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b0bb7444-4944-4ed6-b482-e08f5b7d1f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:12:59.861965 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1061/2124] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:12:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:12:59.864340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53893502-05e8-4cf8-9439-c9a69d0fe17e None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:59.905000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53893502-05e8-4cf8-9439-c9a69d0fe17e None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:12:59.931741 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf65ca2-b40e-45aa-a40f-2bcae10e5f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:12:59.932985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf65ca2-b40e-45aa-a40f-2bcae10e5f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:12:59.933435 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf65ca2-b40e-45aa-a40f-2bcae10e5f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:12:59.933912 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf65ca2-b40e-45aa-a40f-2bcae10e5f92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:12:59.936238 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf65ca2-b40e-45aa-a40f-2bcae10e5f92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:12:59.938408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf65ca2-b40e-45aa-a40f-2bcae10e5f92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:12:59.938865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf65ca2-b40e-45aa-a40f-2bcae10e5f92 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:12:59.959837 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1064/2125] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:12:59.965384 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb7444-4944-4ed6-b482-e08f5b7d1f44 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:12:59.977541 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb7444-4944-4ed6-b482-e08f5b7d1f44 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:00.002970 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc046d-1af5-4562-b09c-f24e994fac04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:00.008540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc046d-1af5-4562-b09c-f24e994fac04 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:00.009684 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc046d-1af5-4562-b09c-f24e994fac04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:00.010697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc046d-1af5-4562-b09c-f24e994fac04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:00.012422 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc046d-1af5-4562-b09c-f24e994fac04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:00.018652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc046d-1af5-4562-b09c-f24e994fac04 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:00.184718 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1062/2126] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:12:59 2022] POST /identity/v3/projects => generated 373 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:13:00.185840 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf65ca2-b40e-45aa-a40f-2bcae10e5f92 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:00.221582 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf65ca2-b40e-45aa-a40f-2bcae10e5f92 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:00.343955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:13:00.447726 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1fda1-46c6-4951-84a9-43d49558ff8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:00.448308 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1fda1-46c6-4951-84a9-43d49558ff8e admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:00.448592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1fda1-46c6-4951-84a9-43d49558ff8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:00.448878 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1fda1-46c6-4951-84a9-43d49558ff8e admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:00.449916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c1fda1-46c6-4951-84a9-43d49558ff8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:00.453232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c1fda1-46c6-4951-84a9-43d49558ff8e admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:00.590807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1065/2127] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:13:00 2022] POST /identity/v3/users => generated 287 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:13:00.595903 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc046d-1af5-4562-b09c-f24e994fac04 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:00.612593 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc046d-1af5-4562-b09c-f24e994fac04 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:00.630446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6f3e9-ec81-4200-bb12-576f138653bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:00.631174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6f3e9-ec81-4200-bb12-576f138653bc admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:00.631651 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6f3e9-ec81-4200-bb12-576f138653bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:00.631949 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6f3e9-ec81-4200-bb12-576f138653bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:00.632787 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a6f3e9-ec81-4200-bb12-576f138653bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:00.634513 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a6f3e9-ec81-4200-bb12-576f138653bc admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:00.651149 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1063/2128] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:13:00.656028 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1fda1-46c6-4951-84a9-43d49558ff8e admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:00.670794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1fda1-46c6-4951-84a9-43d49558ff8e admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:00.686814 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7871b-9e97-4e8b-aff0-77a917aceba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:00.687646 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7871b-9e97-4e8b-aff0-77a917aceba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:00.687937 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7871b-9e97-4e8b-aff0-77a917aceba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:00.688219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7871b-9e97-4e8b-aff0-77a917aceba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:00.688899 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:13:00.689152 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7871b-9e97-4e8b-aff0-77a917aceba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:00.698496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7871b-9e97-4e8b-aff0-77a917aceba0 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:00.728841 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1066/2129] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:13:00.731840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6f3e9-ec81-4200-bb12-576f138653bc admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:00.767757 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6f3e9-ec81-4200-bb12-576f138653bc admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:00.794236 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d880e-efcf-49ae-bb6f-9475c6e07ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:00.796032 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d880e-efcf-49ae-bb6f-9475c6e07ddb admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:00.799455 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d880e-efcf-49ae-bb6f-9475c6e07ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:00.799845 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d880e-efcf-49ae-bb6f-9475c6e07ddb admin admin] PATH_INFO: `/v3/projects/ae0c6ef645cc494798e34b86e955bf91/users/d420dccecf4348eeb284d66eb77512f9/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:00.845963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d880e-efcf-49ae-bb6f-9475c6e07ddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae0c6ef645cc494798e34b86e955bf91, user_id=d420dccecf4348eeb284d66eb77512f9, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:00.851344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d880e-efcf-49ae-bb6f-9475c6e07ddb admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:01.042299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1064/2130] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:13:00 2022] PUT /identity/v3/projects/ae0c6ef645cc494798e34b86e955bf91/users/d420dccecf4348eeb284d66eb77512f9/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:13:01.073238 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3d70f-0e83-4586-985f-15e86f728419 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:01.073828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3d70f-0e83-4586-985f-15e86f728419 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:01.074381 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3d70f-0e83-4586-985f-15e86f728419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:01.193501 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-ecc3d70f-0e83-4586-985f-15e86f728419 None None] MFA Rules not processed for user `d420dccecf4348eeb284d66eb77512f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:01.195859 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ecc3d70f-0e83-4586-985f-15e86f728419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:01.463200 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1067/2131] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:13:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:13:01.481742 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d880e-efcf-49ae-bb6f-9475c6e07ddb admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:01.484476 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-272d880e-efcf-49ae-bb6f-9475c6e07ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:01.620898 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d880e-efcf-49ae-bb6f-9475c6e07ddb admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:01.630114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fc675-be02-4b77-86f2-0d6ec41dbf4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:01.631575 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fc675-be02-4b77-86f2-0d6ec41dbf4d service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:01.632584 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fc675-be02-4b77-86f2-0d6ec41dbf4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:01.633612 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fc675-be02-4b77-86f2-0d6ec41dbf4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:01.663086 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234fc675-be02-4b77-86f2-0d6ec41dbf4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:01.673437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234fc675-be02-4b77-86f2-0d6ec41dbf4d service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:01.729113 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1065/2132] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:13:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:13:02.472451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df2512-f9c0-4c92-9000-5d74c69d46a4 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:02.474956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df2512-f9c0-4c92-9000-5d74c69d46a4 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:02.476611 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df2512-f9c0-4c92-9000-5d74c69d46a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:02.572308 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-a6df2512-f9c0-4c92-9000-5d74c69d46a4 None None] MFA Rules not processed for user `096fd1b2e2b4482da1026d3fb7e192c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:02.575274 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a6df2512-f9c0-4c92-9000-5d74c69d46a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:02.655232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:13:02.683695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1068/2133] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:13:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:13:03.331319 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:13:04.967339 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944421d0-4df7-460a-9822-bfa3157da415 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:04.967982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944421d0-4df7-460a-9822-bfa3157da415 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:04.967982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944421d0-4df7-460a-9822-bfa3157da415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:05.008720 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-944421d0-4df7-460a-9822-bfa3157da415 None None] MFA Rules not processed for user `a4f499707998403d8f833eb783599466`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:05.010279 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-944421d0-4df7-460a-9822-bfa3157da415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:05.148883 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1066/2134] 10.222.0.27 () {58 vars in 1009 bytes} [Sat Jul 16 15:13:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:13:05.305294 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df2512-f9c0-4c92-9000-5d74c69d46a4 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:05.311219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a6df2512-f9c0-4c92-9000-5d74c69d46a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:05.587219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df2512-f9c0-4c92-9000-5d74c69d46a4 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:05.604506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628055a-9de8-4fe7-88c6-db7c20be0957 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1464c83fb294a3d90e5763ef648e834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:05.605261 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628055a-9de8-4fe7-88c6-db7c20be0957 service swift] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:05.609122 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628055a-9de8-4fe7-88c6-db7c20be0957 service swift] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:05.609122 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628055a-9de8-4fe7-88c6-db7c20be0957 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:05.634807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628055a-9de8-4fe7-88c6-db7c20be0957 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:05.647455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628055a-9de8-4fe7-88c6-db7c20be0957 service swift] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:05.689216 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1069/2135] 10.222.0.27 () {60 vars in 1384 bytes} [Sat Jul 16 15:13:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 15:13:14.924821 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e1f66-274d-4c16-b55c-0abe56b912a9 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:14.927324 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e1f66-274d-4c16-b55c-0abe56b912a9 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:14.927324 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e1f66-274d-4c16-b55c-0abe56b912a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:14.953676 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:13:15.018794 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d46e1f66-274d-4c16-b55c-0abe56b912a9 None None] MFA Rules not processed for user `e41aae123ec64a12a614635f23368cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:15.020338 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d46e1f66-274d-4c16-b55c-0abe56b912a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:15.254711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1067/2136] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:13:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:13:15.260800 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628055a-9de8-4fe7-88c6-db7c20be0957 service swift] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:15.262315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-0628055a-9de8-4fe7-88c6-db7c20be0957 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:15.565505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628055a-9de8-4fe7-88c6-db7c20be0957 service swift] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:15.575138 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a7e8c-86eb-4283-8487-0400630a9fa8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:15.576573 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a7e8c-86eb-4283-8487-0400630a9fa8 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:15.576812 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a7e8c-86eb-4283-8487-0400630a9fa8 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:15.577092 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a7e8c-86eb-4283-8487-0400630a9fa8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:15.579164 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-573a7e8c-86eb-4283-8487-0400630a9fa8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:15.826480 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a7e8c-86eb-4283-8487-0400630a9fa8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:15.859604 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a7e8c-86eb-4283-8487-0400630a9fa8 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:15.946562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1070/2137] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:13:15 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 686 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:13:19.871628 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e1f66-274d-4c16-b55c-0abe56b912a9 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:20.119390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e1f66-274d-4c16-b55c-0abe56b912a9 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:20.135253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a101b-0e1f-457e-82ed-6f6be55a837e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:20.136525 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a101b-0e1f-457e-82ed-6f6be55a837e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:20.137629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a101b-0e1f-457e-82ed-6f6be55a837e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:20.138205 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a101b-0e1f-457e-82ed-6f6be55a837e admin admin] PATH_INFO: `/v3/users/d420dccecf4348eeb284d66eb77512f9` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:20.145970 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a101b-0e1f-457e-82ed-6f6be55a837e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d420dccecf4348eeb284d66eb77512f9)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:20.151665 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a101b-0e1f-457e-82ed-6f6be55a837e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:20.401850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-bc3a101b-0e1f-457e-82ed-6f6be55a837e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd420dccecf4348eeb284d66eb77512f9'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:20.479838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-bc3a101b-0e1f-457e-82ed-6f6be55a837e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd420dccecf4348eeb284d66eb77512f9'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:20.480949 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-bc3a101b-0e1f-457e-82ed-6f6be55a837e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd420dccecf4348eeb284d66eb77512f9'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:20.491917 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-bc3a101b-0e1f-457e-82ed-6f6be55a837e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd420dccecf4348eeb284d66eb77512f9'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:20.503317 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-bc3a101b-0e1f-457e-82ed-6f6be55a837e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd420dccecf4348eeb284d66eb77512f9'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:20.552347 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1068/2138] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:13:19 2022] DELETE /identity/v3/users/d420dccecf4348eeb284d66eb77512f9 => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:13:20.554052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75071393-1cef-4738-8eb4-ea3050f8f1b2 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:20.554536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75071393-1cef-4738-8eb4-ea3050f8f1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:20.554618 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75071393-1cef-4738-8eb4-ea3050f8f1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:20.580211 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-75071393-1cef-4738-8eb4-ea3050f8f1b2 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:20.583011 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-75071393-1cef-4738-8eb4-ea3050f8f1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:20.620809 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:13:20.711210 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1071/2139] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:13:20.727674 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a101b-0e1f-457e-82ed-6f6be55a837e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:20.730545 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bc3a101b-0e1f-457e-82ed-6f6be55a837e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:20.756511 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:13:20.859415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfe80f-2a01-4654-9603-279f5b8136b0 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:20.859845 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfe80f-2a01-4654-9603-279f5b8136b0 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:20.860332 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfe80f-2a01-4654-9603-279f5b8136b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:20.938331 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-4dcfe80f-2a01-4654-9603-279f5b8136b0 None None] MFA Rules not processed for user `2517f9ee6c3b4d52b44d5da510e7cbf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:20.942755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-4dcfe80f-2a01-4654-9603-279f5b8136b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:21.065004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a101b-0e1f-457e-82ed-6f6be55a837e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:21.131804 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d42a6-99a7-4144-95c3-3925b476ec33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:21.139018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d42a6-99a7-4144-95c3-3925b476ec33 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:21.140095 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d42a6-99a7-4144-95c3-3925b476ec33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:21.140716 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d42a6-99a7-4144-95c3-3925b476ec33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:21.197551 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d42a6-99a7-4144-95c3-3925b476ec33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:21.202625 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d42a6-99a7-4144-95c3-3925b476ec33 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:21.257813 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1072/2140] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:13:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:13:21.269934 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1069/2141] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:13:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:13:22.300378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f8ebc-9077-402e-8f8f-2b17356e0f69 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:22.306014 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f8ebc-9077-402e-8f8f-2b17356e0f69 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:22.306014 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f8ebc-9077-402e-8f8f-2b17356e0f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:22.331302 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-7b3f8ebc-9077-402e-8f8f-2b17356e0f69 None None] MFA Rules not processed for user `ff8ae9f8a9db47fdbc5e63fffb2f78b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:22.338690 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7b3f8ebc-9077-402e-8f8f-2b17356e0f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:22.463734 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1073/2142] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:13:22 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:13:22.618020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d42a6-99a7-4144-95c3-3925b476ec33 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:22.620691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-294d42a6-99a7-4144-95c3-3925b476ec33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:22.666577 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d42a6-99a7-4144-95c3-3925b476ec33 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:22.692770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d4d22-d64b-4dee-a073-8ffb44a3cf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:22.697462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d4d22-d64b-4dee-a073-8ffb44a3cf57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:22.698046 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d4d22-d64b-4dee-a073-8ffb44a3cf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:22.700461 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d4d22-d64b-4dee-a073-8ffb44a3cf57 admin admin] PATH_INFO: `/v3/projects/ae0c6ef645cc494798e34b86e955bf91` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:22.707051 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d4d22-d64b-4dee-a073-8ffb44a3cf57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae0c6ef645cc494798e34b86e955bf91)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:22.710328 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d4d22-d64b-4dee-a073-8ffb44a3cf57 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:22.994770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c02d4d22-d64b-4dee-a073-8ffb44a3cf57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae0c6ef645cc494798e34b86e955bf91', 'request_id': 'req-c02d4d22-d64b-4dee-a073-8ffb44a3cf57'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:23.012172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c02d4d22-d64b-4dee-a073-8ffb44a3cf57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae0c6ef645cc494798e34b86e955bf91', 'request_id': 'req-c02d4d22-d64b-4dee-a073-8ffb44a3cf57'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:23.031745 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c02d4d22-d64b-4dee-a073-8ffb44a3cf57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae0c6ef645cc494798e34b86e955bf91', 'request_id': 'req-c02d4d22-d64b-4dee-a073-8ffb44a3cf57'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:23.199679 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c02d4d22-d64b-4dee-a073-8ffb44a3cf57 admin admin] The token cache is being invalidate because project ae0c6ef645cc494798e34b86e955bf91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:13:23.199970 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c02d4d22-d64b-4dee-a073-8ffb44a3cf57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:23.202532 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c02d4d22-d64b-4dee-a073-8ffb44a3cf57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:23.205103 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1070/2143] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:13:22 2022] DELETE /identity/v3/projects/ae0c6ef645cc494798e34b86e955bf91 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:13:32.446414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=69 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:13:33.198618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:13:37.036563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3984c-f4dc-43a6-aa4e-39885b27c370 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:37.037306 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3984c-f4dc-43a6-aa4e-39885b27c370 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:37.037306 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d3984c-f4dc-43a6-aa4e-39885b27c370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:37.056191 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-55d3984c-f4dc-43a6-aa4e-39885b27c370 None None] MFA Rules not processed for user `a4f499707998403d8f833eb783599466`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:37.057447 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-55d3984c-f4dc-43a6-aa4e-39885b27c370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:37.123839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1074/2144] 10.222.0.27 () {58 vars in 1009 bytes} [Sat Jul 16 15:13:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:13:37.136325 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d4d22-d64b-4dee-a073-8ffb44a3cf57 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:37.138742 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c02d4d22-d64b-4dee-a073-8ffb44a3cf57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:37.216902 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d4d22-d64b-4dee-a073-8ffb44a3cf57 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:37.222071 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be64a90-ce09-4c9a-9850-f24df3b81554 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1464c83fb294a3d90e5763ef648e834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:37.222730 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be64a90-ce09-4c9a-9850-f24df3b81554 service swift] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:37.223038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be64a90-ce09-4c9a-9850-f24df3b81554 service swift] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:37.223327 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be64a90-ce09-4c9a-9850-f24df3b81554 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:37.232300 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be64a90-ce09-4c9a-9850-f24df3b81554 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:37.233888 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be64a90-ce09-4c9a-9850-f24df3b81554 service swift] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:37.243192 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1071/2145] 10.222.0.27 () {60 vars in 1384 bytes} [Sat Jul 16 15:13:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 15:13:37.604939 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3984c-f4dc-43a6-aa4e-39885b27c370 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:37.607965 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-55d3984c-f4dc-43a6-aa4e-39885b27c370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:37.766616 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d3984c-f4dc-43a6-aa4e-39885b27c370 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:37.776583 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b71e5-8e03-4309-b66e-e205d647691c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:37.779095 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b71e5-8e03-4309-b66e-e205d647691c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:37.779626 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b71e5-8e03-4309-b66e-e205d647691c admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:37.780355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b71e5-8e03-4309-b66e-e205d647691c admin admin] PATH_INFO: `/v3/users/b9b9c3168d844fdc9f57475fdcca34ac` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:37.783468 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b71e5-8e03-4309-b66e-e205d647691c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9b9c3168d844fdc9f57475fdcca34ac)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:37.788533 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2b71e5-8e03-4309-b66e-e205d647691c admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:38.026430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-1f2b71e5-8e03-4309-b66e-e205d647691c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9b9c3168d844fdc9f57475fdcca34ac'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:38.102353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-1f2b71e5-8e03-4309-b66e-e205d647691c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9b9c3168d844fdc9f57475fdcca34ac'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:38.117784 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-1f2b71e5-8e03-4309-b66e-e205d647691c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9b9c3168d844fdc9f57475fdcca34ac'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:38.141368 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-1f2b71e5-8e03-4309-b66e-e205d647691c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9b9c3168d844fdc9f57475fdcca34ac'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:38.142563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-1f2b71e5-8e03-4309-b66e-e205d647691c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9b9c3168d844fdc9f57475fdcca34ac'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:38.157798 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11deaa75-03e8-4904-9d59-8100d0d9dc23 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:38.158340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11deaa75-03e8-4904-9d59-8100d0d9dc23 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:38.158526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11deaa75-03e8-4904-9d59-8100d0d9dc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:38.160145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1075/2146] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:13:37 2022] DELETE /identity/v3/users/b9b9c3168d844fdc9f57475fdcca34ac => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:13:38.195447 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-11deaa75-03e8-4904-9d59-8100d0d9dc23 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:38.198148 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-11deaa75-03e8-4904-9d59-8100d0d9dc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:38.366293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1072/2147] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:13:38.380783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b71e5-8e03-4309-b66e-e205d647691c admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:38.382556 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-1f2b71e5-8e03-4309-b66e-e205d647691c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:38.426349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:13:38.500769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b71e5-8e03-4309-b66e-e205d647691c admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:38.511160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf367229-662f-4427-9936-77cd42f3c39c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:38.512537 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf367229-662f-4427-9936-77cd42f3c39c service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:38.513125 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf367229-662f-4427-9936-77cd42f3c39c service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:38.513696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf367229-662f-4427-9936-77cd42f3c39c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:38.534645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf367229-662f-4427-9936-77cd42f3c39c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:38.537614 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf367229-662f-4427-9936-77cd42f3c39c service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:38.559674 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1076/2148] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:13:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:13:38.695007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11deaa75-03e8-4904-9d59-8100d0d9dc23 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:38.698649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-11deaa75-03e8-4904-9d59-8100d0d9dc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:38.852386 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11deaa75-03e8-4904-9d59-8100d0d9dc23 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:38.867684 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349afaf-1f74-4b4a-aaec-3ed59d5e6e1d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a30cee766814c8685578819261705ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:38.870111 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349afaf-1f74-4b4a-aaec-3ed59d5e6e1d service placement] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:38.870984 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349afaf-1f74-4b4a-aaec-3ed59d5e6e1d service placement] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:38.882017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349afaf-1f74-4b4a-aaec-3ed59d5e6e1d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:38.882017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2349afaf-1f74-4b4a-aaec-3ed59d5e6e1d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:38.922459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2349afaf-1f74-4b4a-aaec-3ed59d5e6e1d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:38.928569 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2349afaf-1f74-4b4a-aaec-3ed59d5e6e1d service placement] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:38.954995 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1073/2149] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:13:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:13:39.666870 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf367229-662f-4427-9936-77cd42f3c39c service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:39.670445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-cf367229-662f-4427-9936-77cd42f3c39c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:39.717162 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf367229-662f-4427-9936-77cd42f3c39c service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:39.732884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd0cee-8796-40e4-97c1-ef42fcdf7837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:39.734100 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd0cee-8796-40e4-97c1-ef42fcdf7837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:39.734534 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd0cee-8796-40e4-97c1-ef42fcdf7837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:39.735110 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd0cee-8796-40e4-97c1-ef42fcdf7837 admin admin] PATH_INFO: `/v3/projects/5982b82cd0eb442aaf4fad9a89e015d4` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:39.737652 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd0cee-8796-40e4-97c1-ef42fcdf7837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5982b82cd0eb442aaf4fad9a89e015d4)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:39.741776 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd0cee-8796-40e4-97c1-ef42fcdf7837 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:40.099579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d1fd0cee-8796-40e4-97c1-ef42fcdf7837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5982b82cd0eb442aaf4fad9a89e015d4', 'request_id': 'req-d1fd0cee-8796-40e4-97c1-ef42fcdf7837'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:40.114753 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d1fd0cee-8796-40e4-97c1-ef42fcdf7837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5982b82cd0eb442aaf4fad9a89e015d4', 'request_id': 'req-d1fd0cee-8796-40e4-97c1-ef42fcdf7837'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:40.139478 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d1fd0cee-8796-40e4-97c1-ef42fcdf7837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5982b82cd0eb442aaf4fad9a89e015d4', 'request_id': 'req-d1fd0cee-8796-40e4-97c1-ef42fcdf7837'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:40.300866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d1fd0cee-8796-40e4-97c1-ef42fcdf7837 admin admin] The token cache is being invalidate because project 5982b82cd0eb442aaf4fad9a89e015d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:13:40.301219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d1fd0cee-8796-40e4-97c1-ef42fcdf7837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:40.302474 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d1fd0cee-8796-40e4-97c1-ef42fcdf7837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:40.307988 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1077/2150] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:13:39 2022] DELETE /identity/v3/projects/5982b82cd0eb442aaf4fad9a89e015d4 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:13:40.315296 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023394e-044a-4dc7-a33a-426d479a3f88 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:40.316496 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023394e-044a-4dc7-a33a-426d479a3f88 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:40.316496 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023394e-044a-4dc7-a33a-426d479a3f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:40.346780 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307e4f7-acbf-406c-9418-26b2be4bc782 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:40.348675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307e4f7-acbf-406c-9418-26b2be4bc782 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:40.349834 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307e4f7-acbf-406c-9418-26b2be4bc782 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:40.353799 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1078/2151] 10.222.0.27 () {54 vars in 921 bytes} [Sat Jul 16 15:13:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:13:40.359776 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-9023394e-044a-4dc7-a33a-426d479a3f88 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:40.361163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9023394e-044a-4dc7-a33a-426d479a3f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:40.375089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f38f06-4d7e-4502-aab4-cfc732d37f6f None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:40.376395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f38f06-4d7e-4502-aab4-cfc732d37f6f None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:40.377130 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f38f06-4d7e-4502-aab4-cfc732d37f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:40.457955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:13:40.467173 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-d7f38f06-4d7e-4502-aab4-cfc732d37f6f None None] MFA Rules not processed for user `3f2e241494244edf8d283e8f44192c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:40.475440 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-d7f38f06-4d7e-4502-aab4-cfc732d37f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:40.721496 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1074/2152] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:13:40.739016 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24959d2e-1a1b-46e5-be35-63c7a1abdc1e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:40.740736 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24959d2e-1a1b-46e5-be35-63c7a1abdc1e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:40.741287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24959d2e-1a1b-46e5-be35-63c7a1abdc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:40.757246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1079/2153] 10.222.0.27 () {58 vars in 1018 bytes} [Sat Jul 16 15:13:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 16 15:13:40.835511 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-24959d2e-1a1b-46e5-be35-63c7a1abdc1e None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:40.838391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-24959d2e-1a1b-46e5-be35-63c7a1abdc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:40.900457 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1075/2154] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:13:40.908692 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f38f06-4d7e-4502-aab4-cfc732d37f6f None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:40.943879 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f38f06-4d7e-4502-aab4-cfc732d37f6f None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:40.956914 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3082e0-0d1f-487f-af26-a29a8803bdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:40.957601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3082e0-0d1f-487f-af26-a29a8803bdba admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:40.957881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3082e0-0d1f-487f-af26-a29a8803bdba admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:40.958157 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3082e0-0d1f-487f-af26-a29a8803bdba admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:40.958933 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3082e0-0d1f-487f-af26-a29a8803bdba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:13:40.960364 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3082e0-0d1f-487f-af26-a29a8803bdba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:40.964115 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3082e0-0d1f-487f-af26-a29a8803bdba admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:40.996149 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24959d2e-1a1b-46e5-be35-63c7a1abdc1e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:40.998725 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-24959d2e-1a1b-46e5-be35-63c7a1abdc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:40.999301 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1080/2155] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:13:41.005133 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3082e0-0d1f-487f-af26-a29a8803bdba admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:41.025734 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3082e0-0d1f-487f-af26-a29a8803bdba admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:41.037289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded23b2-c5c0-4b99-96f8-e12b4a578082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:41.039886 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded23b2-c5c0-4b99-96f8-e12b4a578082 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:41.039886 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded23b2-c5c0-4b99-96f8-e12b4a578082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:41.039886 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded23b2-c5c0-4b99-96f8-e12b4a578082 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:41.039886 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded23b2-c5c0-4b99-96f8-e12b4a578082 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:41.041657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded23b2-c5c0-4b99-96f8-e12b4a578082 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:41.135712 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24959d2e-1a1b-46e5-be35-63c7a1abdc1e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:41.143409 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c13f92-1f79-43c4-a422-5340012172d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d689162291f4f4b97e886ea20d3a8b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:41.144195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c13f92-1f79-43c4-a422-5340012172d7 service glance] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:41.144544 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c13f92-1f79-43c4-a422-5340012172d7 service glance] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:41.144905 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c13f92-1f79-43c4-a422-5340012172d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:41.158859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c13f92-1f79-43c4-a422-5340012172d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:41.160740 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c13f92-1f79-43c4-a422-5340012172d7 service glance] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:41.189518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1076/2156] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:13:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:13:41.237138 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1081/2157] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:13:41 2022] POST /identity/v3/projects => generated 371 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:13:41.243157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c13f92-1f79-43c4-a422-5340012172d7 service glance] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:41.271644 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c13f92-1f79-43c4-a422-5340012172d7 service glance] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:41.471988 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeae72e-dffe-441b-b63e-c62e45ed0497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:41.472694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeae72e-dffe-441b-b63e-c62e45ed0497 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:41.473055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeae72e-dffe-441b-b63e-c62e45ed0497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:41.473404 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeae72e-dffe-441b-b63e-c62e45ed0497 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:41.474180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeae72e-dffe-441b-b63e-c62e45ed0497 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:41.476325 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeae72e-dffe-441b-b63e-c62e45ed0497 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:41.517558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4147d73-f637-459c-bc0a-014047f4e717 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:41.518212 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4147d73-f637-459c-bc0a-014047f4e717 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:41.518497 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4147d73-f637-459c-bc0a-014047f4e717 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:41.520804 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1082/2158] 10.222.0.27 () {54 vars in 921 bytes} [Sat Jul 16 15:13:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:13:41.529991 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c310bef-961b-40ef-ab61-4ce5be91dcb3 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:41.530280 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c310bef-961b-40ef-ab61-4ce5be91dcb3 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:41.530611 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c310bef-961b-40ef-ab61-4ce5be91dcb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:41.552178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:13:41.598695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1077/2159] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:13:41 2022] POST /identity/v3/users => generated 286 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:13:41.599165 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-3c310bef-961b-40ef-ab61-4ce5be91dcb3 None None] MFA Rules not processed for user `6e372cc62af24664b01a967435eb8902`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:41.601212 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3c310bef-961b-40ef-ab61-4ce5be91dcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:41.613124 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeae72e-dffe-441b-b63e-c62e45ed0497 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:41.647121 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeae72e-dffe-441b-b63e-c62e45ed0497 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:41.666747 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0c40e-d2f4-4474-b4a0-cdf0cc2b1024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:41.668969 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0c40e-d2f4-4474-b4a0-cdf0cc2b1024 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:41.670706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0c40e-d2f4-4474-b4a0-cdf0cc2b1024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:41.671177 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0c40e-d2f4-4474-b4a0-cdf0cc2b1024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:41.684618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0c40e-d2f4-4474-b4a0-cdf0cc2b1024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:41.688989 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0c40e-d2f4-4474-b4a0-cdf0cc2b1024 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:41.742414 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1078/2160] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:13:41.747105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1083/2161] 10.222.0.27 () {58 vars in 1018 bytes} [Sat Jul 16 15:13:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:13:41.747261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0c40e-d2f4-4474-b4a0-cdf0cc2b1024 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:41.747352 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c310bef-961b-40ef-ab61-4ce5be91dcb3 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:41.747352 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3c310bef-961b-40ef-ab61-4ce5be91dcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:41.759911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c310bef-961b-40ef-ab61-4ce5be91dcb3 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:41.764683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb94dcd-a6f0-4236-83d9-8715885a524f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:41.766242 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb94dcd-a6f0-4236-83d9-8715885a524f service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:41.766493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb94dcd-a6f0-4236-83d9-8715885a524f service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:41.766748 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb94dcd-a6f0-4236-83d9-8715885a524f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:41.784777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb94dcd-a6f0-4236-83d9-8715885a524f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:41.786624 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb94dcd-a6f0-4236-83d9-8715885a524f service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:41.791618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0c40e-d2f4-4474-b4a0-cdf0cc2b1024 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:41.807530 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1084/2162] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:13:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:13:41.811964 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff545c8-266d-4a12-820e-dbd9a4436989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:41.811964 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff545c8-266d-4a12-820e-dbd9a4436989 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:41.811964 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff545c8-266d-4a12-820e-dbd9a4436989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:41.812458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff545c8-266d-4a12-820e-dbd9a4436989 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:41.812927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff545c8-266d-4a12-820e-dbd9a4436989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:41.814706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff545c8-266d-4a12-820e-dbd9a4436989 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:41.840919 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb94dcd-a6f0-4236-83d9-8715885a524f service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:41.842709 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1079/2163] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:13:41.864597 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb94dcd-a6f0-4236-83d9-8715885a524f service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:41.882706 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b308b2b-3135-40b7-a01c-7afd6068e6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:41.884071 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b308b2b-3135-40b7-a01c-7afd6068e6bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:41.884891 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b308b2b-3135-40b7-a01c-7afd6068e6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:41.886538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b308b2b-3135-40b7-a01c-7afd6068e6bb admin admin] PATH_INFO: `/v3/projects/c2e5dbe3b92d4ebfa6f9bbe7935d6676/users/9fea6c3e3ccf43bd9850fb74a407f55a/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:41.932930 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ebdd4-cbd4-4ae3-b29c-d93769417f36 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:41.933495 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ebdd4-cbd4-4ae3-b29c-d93769417f36 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:41.934852 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ebdd4-cbd4-4ae3-b29c-d93769417f36 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:41.937771 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1080/2164] 10.222.0.27 () {54 vars in 921 bytes} [Sat Jul 16 15:13:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:13:41.952555 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cbcae-9b4e-4449-b3dc-a719e9a9dbcb None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:41.952864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b308b2b-3135-40b7-a01c-7afd6068e6bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e5dbe3b92d4ebfa6f9bbe7935d6676, user_id=9fea6c3e3ccf43bd9850fb74a407f55a, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:41.953521 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cbcae-9b4e-4449-b3dc-a719e9a9dbcb None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:41.953802 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cbcae-9b4e-4449-b3dc-a719e9a9dbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:41.955933 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b308b2b-3135-40b7-a01c-7afd6068e6bb admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:41.962391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:13:41.978352 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-dc7cbcae-9b4e-4449-b3dc-a719e9a9dbcb None None] MFA Rules not processed for user `3f2e241494244edf8d283e8f44192c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:41.982223 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-dc7cbcae-9b4e-4449-b3dc-a719e9a9dbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:42.023794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1085/2165] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:13:41 2022] PUT /identity/v3/projects/c2e5dbe3b92d4ebfa6f9bbe7935d6676/users/9fea6c3e3ccf43bd9850fb74a407f55a/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:13:42.038404 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8aee7-b82f-46cc-ba08-9589e875a3c0 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:42.039078 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8aee7-b82f-46cc-ba08-9589e875a3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:42.039580 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8aee7-b82f-46cc-ba08-9589e875a3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:42.078164 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-8cb8aee7-b82f-46cc-ba08-9589e875a3c0 None None] MFA Rules not processed for user `9fea6c3e3ccf43bd9850fb74a407f55a`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:42.080944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-8cb8aee7-b82f-46cc-ba08-9589e875a3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:42.145035 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1081/2166] 10.222.0.27 () {58 vars in 1018 bytes} [Sat Jul 16 15:13:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:13:42.166582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cbcae-9b4e-4449-b3dc-a719e9a9dbcb None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:42.177173 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cbcae-9b4e-4449-b3dc-a719e9a9dbcb None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:42.195387 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1086/2167] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:13:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:13:42.212421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8aee7-b82f-46cc-ba08-9589e875a3c0 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:42.214198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-8cb8aee7-b82f-46cc-ba08-9589e875a3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:42.285018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a054a7f3-3a9a-462b-b11b-94251d3b42f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:42.285732 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054a7f3-3a9a-462b-b11b-94251d3b42f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:42.286020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054a7f3-3a9a-462b-b11b-94251d3b42f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:42.288200 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a054a7f3-3a9a-462b-b11b-94251d3b42f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:42.359882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8aee7-b82f-46cc-ba08-9589e875a3c0 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:42.364609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce729d-7c60-4269-abe0-57167b8a6ccf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:42.365474 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce729d-7c60-4269-abe0-57167b8a6ccf service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:42.365944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce729d-7c60-4269-abe0-57167b8a6ccf service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:42.366423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce729d-7c60-4269-abe0-57167b8a6ccf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:42.381606 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce729d-7c60-4269-abe0-57167b8a6ccf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:42.385199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce729d-7c60-4269-abe0-57167b8a6ccf service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:42.404243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1087/2168] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:13:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:13:42.410563 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a054a7f3-3a9a-462b-b11b-94251d3b42f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:42.412025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a054a7f3-3a9a-462b-b11b-94251d3b42f4 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:42.425998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1082/2169] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:13:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:13:42.486404 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce729d-7c60-4269-abe0-57167b8a6ccf service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:42.503144 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:13:42.677298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce729d-7c60-4269-abe0-57167b8a6ccf service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:42.701610 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4e3e0-e8f8-4695-9392-461a293254b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:42.702667 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4e3e0-e8f8-4695-9392-461a293254b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:42.703174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4e3e0-e8f8-4695-9392-461a293254b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:42.703812 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4e3e0-e8f8-4695-9392-461a293254b6 admin admin] PATH_INFO: `/v3/users/9fea6c3e3ccf43bd9850fb74a407f55a` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:42.719208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4e3e0-e8f8-4695-9392-461a293254b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fea6c3e3ccf43bd9850fb74a407f55a)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:42.719793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4e3e0-e8f8-4695-9392-461a293254b6 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:43.138621 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c4c4e3e0-e8f8-4695-9392-461a293254b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fea6c3e3ccf43bd9850fb74a407f55a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:43.250190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c4c4e3e0-e8f8-4695-9392-461a293254b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fea6c3e3ccf43bd9850fb74a407f55a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:43.262624 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c4c4e3e0-e8f8-4695-9392-461a293254b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fea6c3e3ccf43bd9850fb74a407f55a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:43.279540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c4c4e3e0-e8f8-4695-9392-461a293254b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fea6c3e3ccf43bd9850fb74a407f55a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:43.280917 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c4c4e3e0-e8f8-4695-9392-461a293254b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fea6c3e3ccf43bd9850fb74a407f55a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:43.288655 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1088/2170] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:13:42 2022] DELETE /identity/v3/users/9fea6c3e3ccf43bd9850fb74a407f55a => generated 0 bytes in 803 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:13:43.294652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0503dd4-d2c0-4771-a44e-77617e7f4f12 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:43.295390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0503dd4-d2c0-4771-a44e-77617e7f4f12 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:43.295390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0503dd4-d2c0-4771-a44e-77617e7f4f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:43.317550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a0503dd4-d2c0-4771-a44e-77617e7f4f12 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:43.320228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a0503dd4-d2c0-4771-a44e-77617e7f4f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:43.377710 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:13:43.426740 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1083/2171] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:13:43.445248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4e3e0-e8f8-4695-9392-461a293254b6 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:43.448342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c4c4e3e0-e8f8-4695-9392-461a293254b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:43.573029 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4e3e0-e8f8-4695-9392-461a293254b6 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:43.579474 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d84b8-ce4e-426d-89b2-e57908f7ddeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:43.580189 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d84b8-ce4e-426d-89b2-e57908f7ddeb service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:43.580601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d84b8-ce4e-426d-89b2-e57908f7ddeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:43.580907 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d84b8-ce4e-426d-89b2-e57908f7ddeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:43.596433 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d84b8-ce4e-426d-89b2-e57908f7ddeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:43.598367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d84b8-ce4e-426d-89b2-e57908f7ddeb service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:43.612406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1089/2172] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:13:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:13:44.319737 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0503dd4-d2c0-4771-a44e-77617e7f4f12 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:44.321497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a0503dd4-d2c0-4771-a44e-77617e7f4f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:44.349364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0503dd4-d2c0-4771-a44e-77617e7f4f12 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:44.357272 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9b207-b3ea-4766-a9ca-eac846d001e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:44.358263 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9b207-b3ea-4766-a9ca-eac846d001e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:44.358439 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9b207-b3ea-4766-a9ca-eac846d001e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:44.359072 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9b207-b3ea-4766-a9ca-eac846d001e4 admin admin] PATH_INFO: `/v3/projects/c2e5dbe3b92d4ebfa6f9bbe7935d6676` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:44.361306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9b207-b3ea-4766-a9ca-eac846d001e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e5dbe3b92d4ebfa6f9bbe7935d6676)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:44.364170 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9b207-b3ea-4766-a9ca-eac846d001e4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:44.607053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-51c9b207-b3ea-4766-a9ca-eac846d001e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e5dbe3b92d4ebfa6f9bbe7935d6676', 'request_id': 'req-51c9b207-b3ea-4766-a9ca-eac846d001e4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:44.620028 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-51c9b207-b3ea-4766-a9ca-eac846d001e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e5dbe3b92d4ebfa6f9bbe7935d6676', 'request_id': 'req-51c9b207-b3ea-4766-a9ca-eac846d001e4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:44.639021 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-51c9b207-b3ea-4766-a9ca-eac846d001e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e5dbe3b92d4ebfa6f9bbe7935d6676', 'request_id': 'req-51c9b207-b3ea-4766-a9ca-eac846d001e4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:44.775379 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-51c9b207-b3ea-4766-a9ca-eac846d001e4 admin admin] The token cache is being invalidate because project c2e5dbe3b92d4ebfa6f9bbe7935d6676 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:13:44.775676 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-51c9b207-b3ea-4766-a9ca-eac846d001e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:44.776914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-51c9b207-b3ea-4766-a9ca-eac846d001e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:13:44.784187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1084/2173] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:13:44 2022] DELETE /identity/v3/projects/c2e5dbe3b92d4ebfa6f9bbe7935d6676 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:13:44.789470 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2569f8f-b0ee-4b82-9675-c093087261e7 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:44.790297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2569f8f-b0ee-4b82-9675-c093087261e7 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:44.790297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2569f8f-b0ee-4b82-9675-c093087261e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:44.813530 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-c2569f8f-b0ee-4b82-9675-c093087261e7 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:44.816576 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c2569f8f-b0ee-4b82-9675-c093087261e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:44.859661 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:13:44.923157 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1090/2174] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:13:44.931760 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a93719-7af6-44c2-a118-094f54c5d9b1 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:44.931760 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a93719-7af6-44c2-a118-094f54c5d9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:44.932262 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a93719-7af6-44c2-a118-094f54c5d9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:44.961361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-22a93719-7af6-44c2-a118-094f54c5d9b1 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:44.963268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-22a93719-7af6-44c2-a118-094f54c5d9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:44.974816 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1085/2175] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:13:44.979820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2569f8f-b0ee-4b82-9675-c093087261e7 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:45.029328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2569f8f-b0ee-4b82-9675-c093087261e7 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:45.042591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eebbe0-f6ef-4880-87a1-0e728bbb1a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:45.046247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eebbe0-f6ef-4880-87a1-0e728bbb1a6c admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:45.046721 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eebbe0-f6ef-4880-87a1-0e728bbb1a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:45.047200 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eebbe0-f6ef-4880-87a1-0e728bbb1a6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:45.048804 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eebbe0-f6ef-4880-87a1-0e728bbb1a6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:13:45.049240 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eebbe0-f6ef-4880-87a1-0e728bbb1a6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:45.051412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eebbe0-f6ef-4880-87a1-0e728bbb1a6c admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:45.070810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1091/2176] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 15:13:45.081556 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a93719-7af6-44c2-a118-094f54c5d9b1 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:45.127289 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a93719-7af6-44c2-a118-094f54c5d9b1 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:45.139116 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4abc6-5a87-447b-80c9-315826882bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:45.140301 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4abc6-5a87-447b-80c9-315826882bf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:45.141079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4abc6-5a87-447b-80c9-315826882bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:45.141562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4abc6-5a87-447b-80c9-315826882bf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:45.143407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4abc6-5a87-447b-80c9-315826882bf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:45.154518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4abc6-5a87-447b-80c9-315826882bf7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:45.341558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1086/2177] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:13:45 2022] POST /identity/v3/projects => generated 379 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:13:45.346664 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eebbe0-f6ef-4880-87a1-0e728bbb1a6c admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:45.367061 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eebbe0-f6ef-4880-87a1-0e728bbb1a6c admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:45.468754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfdfec1-268d-46c6-aa76-c7b2b08bc228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:45.469648 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfdfec1-268d-46c6-aa76-c7b2b08bc228 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:45.470118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfdfec1-268d-46c6-aa76-c7b2b08bc228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:45.470379 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfdfec1-268d-46c6-aa76-c7b2b08bc228 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:45.471549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfdfec1-268d-46c6-aa76-c7b2b08bc228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:45.474085 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfdfec1-268d-46c6-aa76-c7b2b08bc228 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:45.638666 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1092/2178] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:13:45 2022] POST /identity/v3/users => generated 290 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:13:45.643402 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4abc6-5a87-447b-80c9-315826882bf7 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:45.663773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4abc6-5a87-447b-80c9-315826882bf7 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:45.677184 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ee80d-9352-4a54-8bcb-a5d7d784ca63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:45.678301 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ee80d-9352-4a54-8bcb-a5d7d784ca63 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:45.678762 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ee80d-9352-4a54-8bcb-a5d7d784ca63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:45.679272 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ee80d-9352-4a54-8bcb-a5d7d784ca63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:45.680706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ee80d-9352-4a54-8bcb-a5d7d784ca63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:45.693050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ee80d-9352-4a54-8bcb-a5d7d784ca63 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:45.721097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1087/2179] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:13:45.726975 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfdfec1-268d-46c6-aa76-c7b2b08bc228 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:45.742569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfdfec1-268d-46c6-aa76-c7b2b08bc228 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:45.767498 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a817570-1dc7-47d4-9e0d-9810989e9a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:45.770457 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a817570-1dc7-47d4-9e0d-9810989e9a6b admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:45.771284 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a817570-1dc7-47d4-9e0d-9810989e9a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:45.771569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a817570-1dc7-47d4-9e0d-9810989e9a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:45.772340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a817570-1dc7-47d4-9e0d-9810989e9a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:45.774422 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a817570-1dc7-47d4-9e0d-9810989e9a6b admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:45.794826 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1093/2180] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:13:45.796669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ee80d-9352-4a54-8bcb-a5d7d784ca63 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:45.818434 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ee80d-9352-4a54-8bcb-a5d7d784ca63 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:45.823581 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a3177-e609-4c22-99f2-e25090fdc79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:45.825301 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a3177-e609-4c22-99f2-e25090fdc79a admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:45.825587 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a3177-e609-4c22-99f2-e25090fdc79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:45.825916 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a3177-e609-4c22-99f2-e25090fdc79a admin admin] PATH_INFO: `/v3/projects/979ae4d7222a4afd84ecd7fb7dc3552f/users/e97b2376802c4e3f936eaa3d33b5b194/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:45.848277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a3177-e609-4c22-99f2-e25090fdc79a admin admin] RBAC: Authorizing `identity:create_grant(project_id=979ae4d7222a4afd84ecd7fb7dc3552f, user_id=e97b2376802c4e3f936eaa3d33b5b194, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:45.851927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a3177-e609-4c22-99f2-e25090fdc79a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:45.945824 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1088/2181] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:13:45 2022] PUT /identity/v3/projects/979ae4d7222a4afd84ecd7fb7dc3552f/users/e97b2376802c4e3f936eaa3d33b5b194/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:13:45.952176 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3a1fa-7e24-438f-84a8-6f8e2603fad0 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:45.952625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3a1fa-7e24-438f-84a8-6f8e2603fad0 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:45.952991 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3a1fa-7e24-438f-84a8-6f8e2603fad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:45.981056 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-44c3a1fa-7e24-438f-84a8-6f8e2603fad0 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:45.983177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-44c3a1fa-7e24-438f-84a8-6f8e2603fad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:46.110539 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1094/2182] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:13:46.119003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a3177-e609-4c22-99f2-e25090fdc79a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:46.124822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-914a3177-e609-4c22-99f2-e25090fdc79a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:46.284721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a3177-e609-4c22-99f2-e25090fdc79a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:46.288982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7adfe14-0ebb-4038-aa4a-89cd23249545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:46.289796 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adfe14-0ebb-4038-aa4a-89cd23249545 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:46.290129 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adfe14-0ebb-4038-aa4a-89cd23249545 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:46.290739 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adfe14-0ebb-4038-aa4a-89cd23249545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:46.304498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7adfe14-0ebb-4038-aa4a-89cd23249545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:46.306174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7adfe14-0ebb-4038-aa4a-89cd23249545 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:46.314117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:13:46.325288 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1089/2183] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:13:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:13:48.626172 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1a686-e638-4c51-8691-6c4388eb6688 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:48.626785 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1a686-e638-4c51-8691-6c4388eb6688 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:48.626785 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1a686-e638-4c51-8691-6c4388eb6688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:48.657798 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-ecb1a686-e638-4c51-8691-6c4388eb6688 None None] MFA Rules not processed for user `a4f499707998403d8f833eb783599466`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:48.659086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ecb1a686-e638-4c51-8691-6c4388eb6688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:48.719479 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:13:48.770293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1095/2184] 10.222.0.27 () {58 vars in 1009 bytes} [Sat Jul 16 15:13:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:13:48.809102 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7adfe14-0ebb-4038-aa4a-89cd23249545 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:48.811746 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d7adfe14-0ebb-4038-aa4a-89cd23249545 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:48.949220 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7adfe14-0ebb-4038-aa4a-89cd23249545 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:48.955877 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87627c34-37c6-4d12-8df6-68245f0c7a6f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1464c83fb294a3d90e5763ef648e834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:48.956548 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87627c34-37c6-4d12-8df6-68245f0c7a6f service swift] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:48.956827 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87627c34-37c6-4d12-8df6-68245f0c7a6f service swift] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:48.957098 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87627c34-37c6-4d12-8df6-68245f0c7a6f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:48.985804 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87627c34-37c6-4d12-8df6-68245f0c7a6f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:48.987530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87627c34-37c6-4d12-8df6-68245f0c7a6f service swift] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:48.998788 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1090/2185] 10.222.0.27 () {60 vars in 1384 bytes} [Sat Jul 16 15:13:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 16 15:13:58.849921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=7 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:13:58.998929 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=32 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:13:59.537144 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c6205-52e2-4d25-b98c-514cc0a1fd2e None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:59.537712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c6205-52e2-4d25-b98c-514cc0a1fd2e None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:59.537895 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c6205-52e2-4d25-b98c-514cc0a1fd2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:59.614725 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-291c6205-52e2-4d25-b98c-514cc0a1fd2e None None] MFA Rules not processed for user `e97b2376802c4e3f936eaa3d33b5b194`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:13:59.616026 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-291c6205-52e2-4d25-b98c-514cc0a1fd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:13:59.764518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1096/2186] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:13:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:13:59.769624 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87627c34-37c6-4d12-8df6-68245f0c7a6f service swift] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:13:59.784005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87627c34-37c6-4d12-8df6-68245f0c7a6f service swift] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:13:59.798954 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de16ccc-f352-4479-97ab-28dbe2246ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:13:59.799649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de16ccc-f352-4479-97ab-28dbe2246ccc admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:13:59.799930 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de16ccc-f352-4479-97ab-28dbe2246ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:13:59.800271 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de16ccc-f352-4479-97ab-28dbe2246ccc admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:13:59.800928 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de16ccc-f352-4479-97ab-28dbe2246ccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:13:59.811291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de16ccc-f352-4479-97ab-28dbe2246ccc admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:13:59.977417 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1091/2187] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:13:59 2022] POST /identity/v3/projects => generated 377 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:13:59.977779 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c6205-52e2-4d25-b98c-514cc0a1fd2e None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:14:00.002067 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c6205-52e2-4d25-b98c-514cc0a1fd2e None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:14:00.275881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a25c3-5232-402c-9751-3b81661ddf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:14:00.277259 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a25c3-5232-402c-9751-3b81661ddf64 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:00.277557 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a25c3-5232-402c-9751-3b81661ddf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:00.277851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a25c3-5232-402c-9751-3b81661ddf64 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:00.278594 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a25c3-5232-402c-9751-3b81661ddf64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:14:00.283971 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005a25c3-5232-402c-9751-3b81661ddf64 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:14:00.468899 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1097/2188] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:13:59 2022] POST /identity/v3/users => generated 295 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:14:00.473498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de16ccc-f352-4479-97ab-28dbe2246ccc admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:14:00.501845 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de16ccc-f352-4479-97ab-28dbe2246ccc admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:14:00.519997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa5cfa-dcf0-4d15-9ef5-9e9a3c7745e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:14:00.530184 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa5cfa-dcf0-4d15-9ef5-9e9a3c7745e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:00.530702 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa5cfa-dcf0-4d15-9ef5-9e9a3c7745e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:00.533864 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa5cfa-dcf0-4d15-9ef5-9e9a3c7745e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:00.535811 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa5cfa-dcf0-4d15-9ef5-9e9a3c7745e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:14:00.554264 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa5cfa-dcf0-4d15-9ef5-9e9a3c7745e1 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:14:00.617239 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a25c3-5232-402c-9751-3b81661ddf64 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:14:00.619880 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1092/2189] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:14:00.638232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a25c3-5232-402c-9751-3b81661ddf64 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:14:00.659611 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaddda16-246c-4901-9c1c-33b0bbd3a1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:14:00.662326 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddda16-246c-4901-9c1c-33b0bbd3a1b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:00.662487 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddda16-246c-4901-9c1c-33b0bbd3a1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:00.662969 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddda16-246c-4901-9c1c-33b0bbd3a1b6 admin admin] PATH_INFO: `/v3/projects/edaadd118ae94e1589641d0f8fc40745/users/1a76a5d15bc1428483ffe66837f914be/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:00.719109 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaddda16-246c-4901-9c1c-33b0bbd3a1b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edaadd118ae94e1589641d0f8fc40745, user_id=1a76a5d15bc1428483ffe66837f914be, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:14:00.722409 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaddda16-246c-4901-9c1c-33b0bbd3a1b6 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:14:00.881883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1098/2190] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:14:00 2022] PUT /identity/v3/projects/edaadd118ae94e1589641d0f8fc40745/users/1a76a5d15bc1428483ffe66837f914be/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:14:00.889594 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa5cfa-dcf0-4d15-9ef5-9e9a3c7745e1 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:14:00.911665 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa5cfa-dcf0-4d15-9ef5-9e9a3c7745e1 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:14:01.136771 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068ead9b-7a8c-4e5c-9667-9932ca97daed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:14:01.143287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ead9b-7a8c-4e5c-9667-9932ca97daed admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:01.143884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ead9b-7a8c-4e5c-9667-9932ca97daed admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:01.144342 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ead9b-7a8c-4e5c-9667-9932ca97daed admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:01.145336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ead9b-7a8c-4e5c-9667-9932ca97daed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:14:01.147764 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068ead9b-7a8c-4e5c-9667-9932ca97daed admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:14:01.187330 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1093/2191] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:14:01.192774 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaddda16-246c-4901-9c1c-33b0bbd3a1b6 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:14:01.234774 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaddda16-246c-4901-9c1c-33b0bbd3a1b6 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:14:01.267397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e563ee-fd9e-4f03-a639-fdef3b3ad412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:14:01.268328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e563ee-fd9e-4f03-a639-fdef3b3ad412 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:01.268618 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e563ee-fd9e-4f03-a639-fdef3b3ad412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:01.268905 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e563ee-fd9e-4f03-a639-fdef3b3ad412 admin admin] PATH_INFO: `/v3/projects/edaadd118ae94e1589641d0f8fc40745/users/1a76a5d15bc1428483ffe66837f914be/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:01.280785 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e563ee-fd9e-4f03-a639-fdef3b3ad412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edaadd118ae94e1589641d0f8fc40745, user_id=1a76a5d15bc1428483ffe66837f914be, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:14:01.284433 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e563ee-fd9e-4f03-a639-fdef3b3ad412 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:14:01.322724 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1099/2192] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:14:01 2022] PUT /identity/v3/projects/edaadd118ae94e1589641d0f8fc40745/users/1a76a5d15bc1428483ffe66837f914be/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:14:08.251452 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee146e6-2869-4935-83d1-8e0ebe5c4ae2 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:08.252293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee146e6-2869-4935-83d1-8e0ebe5c4ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:08.252293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee146e6-2869-4935-83d1-8e0ebe5c4ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:08.299818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-bee146e6-2869-4935-83d1-8e0ebe5c4ae2 None None] MFA Rules not processed for user `a4f499707998403d8f833eb783599466`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:14:08.303936 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bee146e6-2869-4935-83d1-8e0ebe5c4ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:14:08.465438 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1094/2193] 10.222.0.27 () {58 vars in 967 bytes} [Sat Jul 16 15:14:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:14:08.488029 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e563ee-fd9e-4f03-a639-fdef3b3ad412 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:14:08.603457 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:14:08.649941 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e563ee-fd9e-4f03-a639-fdef3b3ad412 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:14:08.657781 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90052792-7b60-4047-b47f-e8aaa9adf972 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1464c83fb294a3d90e5763ef648e834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:14:08.658482 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90052792-7b60-4047-b47f-e8aaa9adf972 service swift] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:08.658859 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90052792-7b60-4047-b47f-e8aaa9adf972 service swift] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:08.659324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90052792-7b60-4047-b47f-e8aaa9adf972 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:08.669670 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90052792-7b60-4047-b47f-e8aaa9adf972 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:14:08.673819 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90052792-7b60-4047-b47f-e8aaa9adf972 service swift] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:14:08.681469 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1100/2194] 10.222.0.27 () {60 vars in 1384 bytes} [Sat Jul 16 15:14:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 15:14:12.877945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee146e6-2869-4935-83d1-8e0ebe5c4ae2 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:14:12.885332 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bee146e6-2869-4935-83d1-8e0ebe5c4ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:14:12.995947 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee146e6-2869-4935-83d1-8e0ebe5c4ae2 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:14:13.003402 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1833ece-e535-4e56-86a9-126f720e4a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:14:13.005013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1833ece-e535-4e56-86a9-126f720e4a97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:13.005508 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1833ece-e535-4e56-86a9-126f720e4a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:13.006005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1833ece-e535-4e56-86a9-126f720e4a97 admin admin] PATH_INFO: `/v3/users/8ba8498f6fbc40c18cadbb4ce6667855` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:13.008687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1833ece-e535-4e56-86a9-126f720e4a97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ba8498f6fbc40c18cadbb4ce6667855)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:14:13.012889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1833ece-e535-4e56-86a9-126f720e4a97 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:14:13.033854 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:14:13.191838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b1833ece-e535-4e56-86a9-126f720e4a97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ba8498f6fbc40c18cadbb4ce6667855'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:14:13.295583 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b1833ece-e535-4e56-86a9-126f720e4a97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ba8498f6fbc40c18cadbb4ce6667855'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:14:13.296428 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b1833ece-e535-4e56-86a9-126f720e4a97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ba8498f6fbc40c18cadbb4ce6667855'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:14:13.297232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b1833ece-e535-4e56-86a9-126f720e4a97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ba8498f6fbc40c18cadbb4ce6667855'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:14:13.306798 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b1833ece-e535-4e56-86a9-126f720e4a97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ba8498f6fbc40c18cadbb4ce6667855'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:14:13.346419 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1095/2195] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:14:12 2022] DELETE /identity/v3/users/8ba8498f6fbc40c18cadbb4ce6667855 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:14:13.498298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11888c3a-3b34-4b38-b6de-b685d2160bf8 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:13.499200 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11888c3a-3b34-4b38-b6de-b685d2160bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:13.499394 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11888c3a-3b34-4b38-b6de-b685d2160bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:13.562782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-11888c3a-3b34-4b38-b6de-b685d2160bf8 None None] MFA Rules not processed for user `1a76a5d15bc1428483ffe66837f914be`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:14:13.565979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-11888c3a-3b34-4b38-b6de-b685d2160bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:14:13.679427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1101/2196] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:14:13 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:14:13.693845 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1833ece-e535-4e56-86a9-126f720e4a97 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:14:13.696490 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b1833ece-e535-4e56-86a9-126f720e4a97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:14:13.817305 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1833ece-e535-4e56-86a9-126f720e4a97 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:14:13.839946 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588767c9-c80a-4f74-9723-26d1c8ecff7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:14:13.839946 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588767c9-c80a-4f74-9723-26d1c8ecff7b service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:13.839946 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588767c9-c80a-4f74-9723-26d1c8ecff7b service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:13.839946 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588767c9-c80a-4f74-9723-26d1c8ecff7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:13.842585 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-588767c9-c80a-4f74-9723-26d1c8ecff7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:14:13.843514 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11888c3a-3b34-4b38-b6de-b685d2160bf8 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:14:13.845758 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-11888c3a-3b34-4b38-b6de-b685d2160bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:14:14.020989 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588767c9-c80a-4f74-9723-26d1c8ecff7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:14:14.024888 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588767c9-c80a-4f74-9723-26d1c8ecff7b service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:14:14.040650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11888c3a-3b34-4b38-b6de-b685d2160bf8 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:14:14.050848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ccbe90-12be-4df0-b8df-55ba4ea095fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:14:14.051573 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccbe90-12be-4df0-b8df-55ba4ea095fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:14.052070 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccbe90-12be-4df0-b8df-55ba4ea095fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:14.052845 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccbe90-12be-4df0-b8df-55ba4ea095fc admin admin] PATH_INFO: `/v3/projects/00e03c837cb7482483081599eed535b1` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:14.059165 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ccbe90-12be-4df0-b8df-55ba4ea095fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=00e03c837cb7482483081599eed535b1)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:14:14.063227 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ccbe90-12be-4df0-b8df-55ba4ea095fc admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:14:14.063948 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1096/2197] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:14:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:14:14.287906 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-95ccbe90-12be-4df0-b8df-55ba4ea095fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00e03c837cb7482483081599eed535b1', 'request_id': 'req-95ccbe90-12be-4df0-b8df-55ba4ea095fc'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:14:14.301508 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-95ccbe90-12be-4df0-b8df-55ba4ea095fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00e03c837cb7482483081599eed535b1', 'request_id': 'req-95ccbe90-12be-4df0-b8df-55ba4ea095fc'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:14:14.303994 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:14:14.320048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-95ccbe90-12be-4df0-b8df-55ba4ea095fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00e03c837cb7482483081599eed535b1', 'request_id': 'req-95ccbe90-12be-4df0-b8df-55ba4ea095fc'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:14:14.470960 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-95ccbe90-12be-4df0-b8df-55ba4ea095fc admin admin] The token cache is being invalidate because project 00e03c837cb7482483081599eed535b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:14:14.470960 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-95ccbe90-12be-4df0-b8df-55ba4ea095fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:14:14.470960 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-95ccbe90-12be-4df0-b8df-55ba4ea095fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:14:14.472893 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1102/2198] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:14:13 2022] DELETE /identity/v3/projects/00e03c837cb7482483081599eed535b1 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:14:14.479997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c8616-baf9-4ab7-aeb6-b941c790b54e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:14.480315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c8616-baf9-4ab7-aeb6-b941c790b54e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:14.480574 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c8616-baf9-4ab7-aeb6-b941c790b54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:14.505152 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-531c8616-baf9-4ab7-aeb6-b941c790b54e None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:14:14.510822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-531c8616-baf9-4ab7-aeb6-b941c790b54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:14:14.538933 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:14:14.600742 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1097/2199] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:14:14.609016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f3590-ed6f-4dd8-b079-e2810c042b6d None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:14.609939 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f3590-ed6f-4dd8-b079-e2810c042b6d None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:14.610408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f3590-ed6f-4dd8-b079-e2810c042b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:14.633238 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-9c2f3590-ed6f-4dd8-b079-e2810c042b6d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:14:14.634933 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-9c2f3590-ed6f-4dd8-b079-e2810c042b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:14:14.653267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c8616-baf9-4ab7-aeb6-b941c790b54e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:14:14.654115 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1103/2200] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:14:14.674217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c8616-baf9-4ab7-aeb6-b941c790b54e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:14:14.686704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b825e8e-2e41-4981-a696-1e15ccf9ba5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:14:14.687190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b825e8e-2e41-4981-a696-1e15ccf9ba5e admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:14.687582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b825e8e-2e41-4981-a696-1e15ccf9ba5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:14.688123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b825e8e-2e41-4981-a696-1e15ccf9ba5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:14.689057 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b825e8e-2e41-4981-a696-1e15ccf9ba5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:14:14.690072 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b825e8e-2e41-4981-a696-1e15ccf9ba5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:14:14.694836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b825e8e-2e41-4981-a696-1e15ccf9ba5e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:14:14.718460 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1098/2201] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:14:14.722569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f3590-ed6f-4dd8-b079-e2810c042b6d None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:14:14.730841 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f3590-ed6f-4dd8-b079-e2810c042b6d None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:14:14.736928 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2598d-a9a7-442e-a126-577b721579e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:14:14.737622 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2598d-a9a7-442e-a126-577b721579e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:14.737905 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2598d-a9a7-442e-a126-577b721579e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:14.738187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2598d-a9a7-442e-a126-577b721579e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:14.739023 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e2598d-a9a7-442e-a126-577b721579e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:14:14.741054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e2598d-a9a7-442e-a126-577b721579e2 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:14:14.896698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1104/2202] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:14:14 2022] POST /identity/v3/projects => generated 381 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:14:14.902714 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b825e8e-2e41-4981-a696-1e15ccf9ba5e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:14:14.919451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b825e8e-2e41-4981-a696-1e15ccf9ba5e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:14:15.012789 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd7117-988a-4a41-a13b-baf44c679ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:14:15.013883 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd7117-988a-4a41-a13b-baf44c679ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:15.014276 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd7117-988a-4a41-a13b-baf44c679ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:15.014683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd7117-988a-4a41-a13b-baf44c679ba2 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:15.015549 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd7117-988a-4a41-a13b-baf44c679ba2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:14:15.018448 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd7117-988a-4a41-a13b-baf44c679ba2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:14:15.184029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1099/2203] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:14:14 2022] POST /identity/v3/users => generated 291 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:14:15.188864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2598d-a9a7-442e-a126-577b721579e2 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:14:15.205935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2598d-a9a7-442e-a126-577b721579e2 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:14:15.215419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ba803-f9da-4371-865c-92f1e222a79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:14:15.216713 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ba803-f9da-4371-865c-92f1e222a79a admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:15.217281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ba803-f9da-4371-865c-92f1e222a79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:15.217787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ba803-f9da-4371-865c-92f1e222a79a admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:15.218617 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339ba803-f9da-4371-865c-92f1e222a79a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:14:15.220839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339ba803-f9da-4371-865c-92f1e222a79a admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:14:15.234410 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1105/2204] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:14:15.239657 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd7117-988a-4a41-a13b-baf44c679ba2 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:14:15.256921 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd7117-988a-4a41-a13b-baf44c679ba2 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:14:15.269179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c6f7e-ed6f-4fcd-8b30-9c2660e78232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:14:15.270335 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c6f7e-ed6f-4fcd-8b30-9c2660e78232 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:15.271034 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c6f7e-ed6f-4fcd-8b30-9c2660e78232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:15.271550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c6f7e-ed6f-4fcd-8b30-9c2660e78232 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:15.272951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c6f7e-ed6f-4fcd-8b30-9c2660e78232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:14:15.277017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c6f7e-ed6f-4fcd-8b30-9c2660e78232 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:14:15.328860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ba803-f9da-4371-865c-92f1e222a79a admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:14:15.336984 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1100/2205] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:14:15.418639 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ba803-f9da-4371-865c-92f1e222a79a admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:14:15.431378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37575f50-0f62-4404-b9e5-eb81d268dc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:14:15.431798 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37575f50-0f62-4404-b9e5-eb81d268dc70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:15.436204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37575f50-0f62-4404-b9e5-eb81d268dc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:15.436204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37575f50-0f62-4404-b9e5-eb81d268dc70 admin admin] PATH_INFO: `/v3/projects/0d5fd17d843c4933b68c201ce7940d5b/users/d9cf1686980749fabd8f54073573ae86/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:15.487055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37575f50-0f62-4404-b9e5-eb81d268dc70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5fd17d843c4933b68c201ce7940d5b, user_id=d9cf1686980749fabd8f54073573ae86, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:14:15.493672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37575f50-0f62-4404-b9e5-eb81d268dc70 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:14:15.633857 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1106/2206] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:14:15 2022] PUT /identity/v3/projects/0d5fd17d843c4933b68c201ce7940d5b/users/d9cf1686980749fabd8f54073573ae86/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:14:15.636956 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0c2a7-a029-4c27-8bcd-a481efa33f3d None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:15.637659 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0c2a7-a029-4c27-8bcd-a481efa33f3d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:15.638293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0c2a7-a029-4c27-8bcd-a481efa33f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:15.666076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-e5f0c2a7-a029-4c27-8bcd-a481efa33f3d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:14:15.667654 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e5f0c2a7-a029-4c27-8bcd-a481efa33f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:14:15.774424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1101/2207] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:14:15.787757 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37575f50-0f62-4404-b9e5-eb81d268dc70 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:14:15.793903 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-37575f50-0f62-4404-b9e5-eb81d268dc70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:14:16.055352 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37575f50-0f62-4404-b9e5-eb81d268dc70 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:14:16.071963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ccc9e-2147-45c1-abe1-5616cde6351a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:14:16.073742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ccc9e-2147-45c1-abe1-5616cde6351a service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:16.074681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ccc9e-2147-45c1-abe1-5616cde6351a service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:16.074681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ccc9e-2147-45c1-abe1-5616cde6351a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:16.108438 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ccc9e-2147-45c1-abe1-5616cde6351a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:14:16.114199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ccc9e-2147-45c1-abe1-5616cde6351a service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:14:16.150298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1107/2208] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:14:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:14:18.248710 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8b972-498a-4d68-ab12-e36ed471460e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:18.249279 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8b972-498a-4d68-ab12-e36ed471460e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:18.250159 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8b972-498a-4d68-ab12-e36ed471460e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:18.297756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-e1c8b972-498a-4d68-ab12-e36ed471460e None None] MFA Rules not processed for user `d9cf1686980749fabd8f54073573ae86`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:14:18.302410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e1c8b972-498a-4d68-ab12-e36ed471460e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:14:18.362310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:14:18.473328 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1102/2209] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:14:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:14:18.491359 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ccc9e-2147-45c1-abe1-5616cde6351a service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:14:18.493201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-217ccc9e-2147-45c1-abe1-5616cde6351a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:14:18.634549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ccc9e-2147-45c1-abe1-5616cde6351a service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:14:18.644594 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b96c4-c34d-41a2-8c1a-bc389d70fee8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:14:18.649896 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b96c4-c34d-41a2-8c1a-bc389d70fee8 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:18.649896 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b96c4-c34d-41a2-8c1a-bc389d70fee8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:18.649896 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b96c4-c34d-41a2-8c1a-bc389d70fee8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:18.676132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b96c4-c34d-41a2-8c1a-bc389d70fee8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:14:18.678263 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b96c4-c34d-41a2-8c1a-bc389d70fee8 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:14:18.694321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1108/2210] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:14:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:14:20.829379 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a362d-a8d1-4dbd-90f2-416dd4a2d35b None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:20.829951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a362d-a8d1-4dbd-90f2-416dd4a2d35b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:20.830157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a362d-a8d1-4dbd-90f2-416dd4a2d35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:20.851440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-615a362d-a8d1-4dbd-90f2-416dd4a2d35b None None] MFA Rules not processed for user `d9cf1686980749fabd8f54073573ae86`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:14:20.852698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-615a362d-a8d1-4dbd-90f2-416dd4a2d35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:14:20.861427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1103/2211] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:14:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:14:21.646355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b96c4-c34d-41a2-8c1a-bc389d70fee8 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:14:21.664843 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b96c4-c34d-41a2-8c1a-bc389d70fee8 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:14:21.677080 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8f746-15d4-49bd-ade9-fb1b83f55f4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:14:21.677863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8f746-15d4-49bd-ade9-fb1b83f55f4b service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:21.678077 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8f746-15d4-49bd-ade9-fb1b83f55f4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:21.678302 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8f746-15d4-49bd-ade9-fb1b83f55f4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:21.680737 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-71d8f746-15d4-49bd-ade9-fb1b83f55f4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:14:21.752683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:14:21.836803 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8f746-15d4-49bd-ade9-fb1b83f55f4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:14:21.840265 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8f746-15d4-49bd-ade9-fb1b83f55f4b service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:14:21.864319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1109/2212] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:14:21 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:14:23.559516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df336c-d226-4384-8f52-e3effe1720e3 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:23.561838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df336c-d226-4384-8f52-e3effe1720e3 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:23.562387 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df336c-d226-4384-8f52-e3effe1720e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:23.592282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-67df336c-d226-4384-8f52-e3effe1720e3 None None] MFA Rules not processed for user `a4f499707998403d8f833eb783599466`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:14:23.593687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-67df336c-d226-4384-8f52-e3effe1720e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:14:23.755973 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1104/2213] 10.222.0.27 () {58 vars in 967 bytes} [Sat Jul 16 15:14:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:14:23.775373 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8f746-15d4-49bd-ade9-fb1b83f55f4b service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:14:23.779769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-71d8f746-15d4-49bd-ade9-fb1b83f55f4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:14:23.933667 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8f746-15d4-49bd-ade9-fb1b83f55f4b service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:14:23.943005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e30bfc-b778-476b-84eb-2b4625023f8c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1464c83fb294a3d90e5763ef648e834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:14:23.943005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e30bfc-b778-476b-84eb-2b4625023f8c service swift] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:23.943005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e30bfc-b778-476b-84eb-2b4625023f8c service swift] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:23.943551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e30bfc-b778-476b-84eb-2b4625023f8c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:23.990422 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e30bfc-b778-476b-84eb-2b4625023f8c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:14:23.993296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e30bfc-b778-476b-84eb-2b4625023f8c service swift] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:14:24.039465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1110/2214] 10.222.0.27 () {60 vars in 1384 bytes} [Sat Jul 16 15:14:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 15:14:33.739374 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=46 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:14:34.008610 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=49 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:14:36.597328 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598d672-7fff-414b-9576-e24903f96952 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:36.598159 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598d672-7fff-414b-9576-e24903f96952 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:36.598159 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598d672-7fff-414b-9576-e24903f96952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:36.621390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-1598d672-7fff-414b-9576-e24903f96952 None None] MFA Rules not processed for user `e97b2376802c4e3f936eaa3d33b5b194`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:14:36.623210 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1598d672-7fff-414b-9576-e24903f96952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:14:36.708182 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1105/2215] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:14:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:14:46.231864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f4c8c-05fc-43cb-afb9-78ead47dccf2 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:14:46.232761 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f4c8c-05fc-43cb-afb9-78ead47dccf2 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:14:46.232761 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f4c8c-05fc-43cb-afb9-78ead47dccf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:14:46.251089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-181f4c8c-05fc-43cb-afb9-78ead47dccf2 None None] MFA Rules not processed for user `096fd1b2e2b4482da1026d3fb7e192c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:14:46.252404 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-181f4c8c-05fc-43cb-afb9-78ead47dccf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:14:46.353378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1111/2216] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:14:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:14:46.700561 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=25 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:14:56.341035 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=25 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:15:07.752045 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598d672-7fff-414b-9576-e24903f96952 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:15:07.762376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598d672-7fff-414b-9576-e24903f96952 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:15:07.766118 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7081eb-ada7-4eb8-bb5e-b5873af4715c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:15:07.766755 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7081eb-ada7-4eb8-bb5e-b5873af4715c service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:15:07.767110 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7081eb-ada7-4eb8-bb5e-b5873af4715c service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:15:07.767348 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7081eb-ada7-4eb8-bb5e-b5873af4715c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:15:07.769112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4b7081eb-ada7-4eb8-bb5e-b5873af4715c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:15:07.839031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7081eb-ada7-4eb8-bb5e-b5873af4715c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:15:07.840506 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7081eb-ada7-4eb8-bb5e-b5873af4715c service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:15:07.851434 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1106/2217] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:15:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:15:13.062535 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502333c-2f46-494f-a345-79fc29a7c183 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:15:13.063080 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502333c-2f46-494f-a345-79fc29a7c183 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:15:13.063180 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502333c-2f46-494f-a345-79fc29a7c183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:15:13.081101 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-a502333c-2f46-494f-a345-79fc29a7c183 None None] MFA Rules not processed for user `2517f9ee6c3b4d52b44d5da510e7cbf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:15:13.082318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a502333c-2f46-494f-a345-79fc29a7c183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:15:13.140088 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1112/2218] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:15:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:15:17.846419 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=30 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:15:23.132934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=25 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:15:53.600611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7081eb-ada7-4eb8-bb5e-b5873af4715c service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:15:53.602092 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4b7081eb-ada7-4eb8-bb5e-b5873af4715c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:15:53.614064 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7081eb-ada7-4eb8-bb5e-b5873af4715c service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:15:53.617665 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e75b6-f280-4d0e-b890-e34fefaacded service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:15:53.618303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e75b6-f280-4d0e-b890-e34fefaacded service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:15:53.618582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e75b6-f280-4d0e-b890-e34fefaacded service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:15:53.618860 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e75b6-f280-4d0e-b890-e34fefaacded service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:15:53.620670 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4d6e75b6-f280-4d0e-b890-e34fefaacded service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:15:53.632519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e75b6-f280-4d0e-b890-e34fefaacded service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:15:53.633947 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e75b6-f280-4d0e-b890-e34fefaacded service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:15:53.644428 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1107/2219] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:15:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:15:54.022049 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502333c-2f46-494f-a345-79fc29a7c183 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:15:54.023531 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a502333c-2f46-494f-a345-79fc29a7c183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:15:54.102200 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502333c-2f46-494f-a345-79fc29a7c183 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:15:54.107785 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e20e8-33de-4db9-993f-6d95cf8f0e59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:15:54.108861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e20e8-33de-4db9-993f-6d95cf8f0e59 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:15:54.109156 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e20e8-33de-4db9-993f-6d95cf8f0e59 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:15:54.109486 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e20e8-33de-4db9-993f-6d95cf8f0e59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:15:54.111302 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-183e20e8-33de-4db9-993f-6d95cf8f0e59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:15:54.171568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e20e8-33de-4db9-993f-6d95cf8f0e59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:15:54.173159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e20e8-33de-4db9-993f-6d95cf8f0e59 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:15:54.182303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1113/2220] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:15:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:15:57.743091 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e75b6-f280-4d0e-b890-e34fefaacded service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:15:57.752338 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e75b6-f280-4d0e-b890-e34fefaacded service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:15:57.756042 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695312ae-2767-4b62-9260-90e70ec03f89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:15:57.756680 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695312ae-2767-4b62-9260-90e70ec03f89 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:15:57.756963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695312ae-2767-4b62-9260-90e70ec03f89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:15:57.757237 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695312ae-2767-4b62-9260-90e70ec03f89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:15:57.759164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-695312ae-2767-4b62-9260-90e70ec03f89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:15:57.771222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695312ae-2767-4b62-9260-90e70ec03f89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:15:57.773449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695312ae-2767-4b62-9260-90e70ec03f89 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:15:57.784601 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1108/2221] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:15:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:15:57.892972 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e20e8-33de-4db9-993f-6d95cf8f0e59 service nova] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:15:57.904090 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e20e8-33de-4db9-993f-6d95cf8f0e59 service nova] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:15:57.909871 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58f2a2-ce08-4d78-9d3a-5760147429cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:15:57.910527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58f2a2-ce08-4d78-9d3a-5760147429cd service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:15:57.910804 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58f2a2-ce08-4d78-9d3a-5760147429cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:15:57.911103 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58f2a2-ce08-4d78-9d3a-5760147429cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:15:57.912857 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-9c58f2a2-ce08-4d78-9d3a-5760147429cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:15:57.926319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58f2a2-ce08-4d78-9d3a-5760147429cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:15:57.927750 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58f2a2-ce08-4d78-9d3a-5760147429cd service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:15:57.939236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1114/2222] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:15:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:16:07.783203 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=12 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:16:07.933678 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=62 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:16:40.676217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695312ae-2767-4b62-9260-90e70ec03f89 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:16:40.677725 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-695312ae-2767-4b62-9260-90e70ec03f89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:16:40.694657 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695312ae-2767-4b62-9260-90e70ec03f89 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:16:40.698395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e98b37-05c2-46fd-9e3b-73ef58fed0a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:16:40.699074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e98b37-05c2-46fd-9e3b-73ef58fed0a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:16:40.699360 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e98b37-05c2-46fd-9e3b-73ef58fed0a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:16:40.699635 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e98b37-05c2-46fd-9e3b-73ef58fed0a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:16:40.701450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-83e98b37-05c2-46fd-9e3b-73ef58fed0a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:16:40.712297 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e98b37-05c2-46fd-9e3b-73ef58fed0a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:16:40.713797 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e98b37-05c2-46fd-9e3b-73ef58fed0a2 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:16:40.724093 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1109/2223] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:16:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:16:44.533054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58f2a2-ce08-4d78-9d3a-5760147429cd service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:16:44.543145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58f2a2-ce08-4d78-9d3a-5760147429cd service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:16:44.547417 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de6de0-c207-4527-b819-6360195925d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:16:44.548055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de6de0-c207-4527-b819-6360195925d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:16:44.548333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de6de0-c207-4527-b819-6360195925d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:16:44.548633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de6de0-c207-4527-b819-6360195925d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:16:44.550392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-14de6de0-c207-4527-b819-6360195925d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:16:44.565558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de6de0-c207-4527-b819-6360195925d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:16:44.567104 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de6de0-c207-4527-b819-6360195925d8 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:16:44.578506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1115/2224] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:16:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:16:50.719072 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=6 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:16:54.573486 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=6 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:16:59.670615 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e98b37-05c2-46fd-9e3b-73ef58fed0a2 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:16:59.719878 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e98b37-05c2-46fd-9e3b-73ef58fed0a2 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:16:59.725975 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a2048-8743-4cee-b1a0-ddc5978bc1ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:16:59.726634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a2048-8743-4cee-b1a0-ddc5978bc1ea service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:16:59.726941 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a2048-8743-4cee-b1a0-ddc5978bc1ea service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:16:59.727217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a2048-8743-4cee-b1a0-ddc5978bc1ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:16:59.729090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-567a2048-8743-4cee-b1a0-ddc5978bc1ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:16:59.769227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a2048-8743-4cee-b1a0-ddc5978bc1ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:16:59.770919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a2048-8743-4cee-b1a0-ddc5978bc1ea service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:16:59.782950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1110/2225] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:16:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:17:09.777652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=28 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:17:19.447830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de6de0-c207-4527-b819-6360195925d8 service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:17:19.534035 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de6de0-c207-4527-b819-6360195925d8 service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:17:19.542195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85614303-27c2-4fe5-b585-eadedb17b3b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:17:19.543061 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85614303-27c2-4fe5-b585-eadedb17b3b9 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:17:19.543353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85614303-27c2-4fe5-b585-eadedb17b3b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:17:19.543650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85614303-27c2-4fe5-b585-eadedb17b3b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:17:19.546557 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-85614303-27c2-4fe5-b585-eadedb17b3b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:17:19.563246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85614303-27c2-4fe5-b585-eadedb17b3b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:17:19.572809 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85614303-27c2-4fe5-b585-eadedb17b3b9 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:17:19.591002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1116/2226] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:17:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:17:29.587344 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=20 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:18:15.485054 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a2048-8743-4cee-b1a0-ddc5978bc1ea service nova] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:18:15.496895 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a2048-8743-4cee-b1a0-ddc5978bc1ea service nova] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:18:15.503763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a89416-c55d-4f1e-b03c-021880806f4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:18:15.504368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a89416-c55d-4f1e-b03c-021880806f4e service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:18:15.504652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a89416-c55d-4f1e-b03c-021880806f4e service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:18:15.504921 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a89416-c55d-4f1e-b03c-021880806f4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:18:15.506743 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-25a89416-c55d-4f1e-b03c-021880806f4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:18:15.575128 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a89416-c55d-4f1e-b03c-021880806f4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:18:15.576743 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a89416-c55d-4f1e-b03c-021880806f4e service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:18:15.587439 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1111/2227] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:18:15 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:18:25.583623 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=28 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:18:42.983887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85614303-27c2-4fe5-b585-eadedb17b3b9 service nova] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:18:42.985505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-85614303-27c2-4fe5-b585-eadedb17b3b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:18:43.095066 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85614303-27c2-4fe5-b585-eadedb17b3b9 service nova] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:18:43.104219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b74e94-ce46-481d-aea3-6f0f1a9ee24e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a30cee766814c8685578819261705ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:18:43.105255 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b74e94-ce46-481d-aea3-6f0f1a9ee24e service placement] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:18:43.105576 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b74e94-ce46-481d-aea3-6f0f1a9ee24e service placement] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:18:43.105855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b74e94-ce46-481d-aea3-6f0f1a9ee24e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:18:43.107780 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-86b74e94-ce46-481d-aea3-6f0f1a9ee24e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:18:43.120653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b74e94-ce46-481d-aea3-6f0f1a9ee24e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:18:43.122262 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b74e94-ce46-481d-aea3-6f0f1a9ee24e service placement] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:18:43.133328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1117/2228] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:18:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:18:53.128980 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=41 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:18:55.927442 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a89416-c55d-4f1e-b03c-021880806f4e service nova] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:18:55.951387 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a89416-c55d-4f1e-b03c-021880806f4e service nova] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:18:55.955380 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f0694-1f1c-4ff7-942a-d5110c6d178a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:18:55.956344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f0694-1f1c-4ff7-942a-d5110c6d178a service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:18:55.956781 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f0694-1f1c-4ff7-942a-d5110c6d178a service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:18:55.957102 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f0694-1f1c-4ff7-942a-d5110c6d178a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:18:55.960045 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-185f0694-1f1c-4ff7-942a-d5110c6d178a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:18:56.027606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f0694-1f1c-4ff7-942a-d5110c6d178a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:18:56.028974 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f0694-1f1c-4ff7-942a-d5110c6d178a service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:18:56.039125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1112/2229] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:18:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:19:06.034245 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=32 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:19:13.451558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b74e94-ce46-481d-aea3-6f0f1a9ee24e service placement] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:19:13.480075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b74e94-ce46-481d-aea3-6f0f1a9ee24e service placement] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:19:13.485602 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef054542-c003-4794-b9f8-b8031b13a867 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:19:13.486242 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef054542-c003-4794-b9f8-b8031b13a867 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:19:13.486530 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef054542-c003-4794-b9f8-b8031b13a867 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:19:13.486811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef054542-c003-4794-b9f8-b8031b13a867 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:19:13.488601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ef054542-c003-4794-b9f8-b8031b13a867 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:19:13.499666 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef054542-c003-4794-b9f8-b8031b13a867 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:19:13.501251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef054542-c003-4794-b9f8-b8031b13a867 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:19:13.511692 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1118/2230] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:19:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:19:21.897290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f0694-1f1c-4ff7-942a-d5110c6d178a service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:19:21.907960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f0694-1f1c-4ff7-942a-d5110c6d178a service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:19:21.911911 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600c904-4649-4590-980c-b3317cccc4e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:19:21.912680 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600c904-4649-4590-980c-b3317cccc4e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:19:21.913052 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600c904-4649-4590-980c-b3317cccc4e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:19:21.913437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600c904-4649-4590-980c-b3317cccc4e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:19:21.923934 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600c904-4649-4590-980c-b3317cccc4e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:19:21.925395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600c904-4649-4590-980c-b3317cccc4e3 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:19:21.935065 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1113/2231] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:19:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:19:23.506200 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=13 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:19:31.930440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=6 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:20:54.051647 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef054542-c003-4794-b9f8-b8031b13a867 service nova] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:20:54.053208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ef054542-c003-4794-b9f8-b8031b13a867 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:20:54.084972 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef054542-c003-4794-b9f8-b8031b13a867 service nova] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:20:54.091413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4ac8c-a7b9-4c06-9560-26bdb389dacc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:20:54.092177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4ac8c-a7b9-4c06-9560-26bdb389dacc service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:20:54.092489 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4ac8c-a7b9-4c06-9560-26bdb389dacc service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:20:54.092898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4ac8c-a7b9-4c06-9560-26bdb389dacc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:20:54.102735 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4ac8c-a7b9-4c06-9560-26bdb389dacc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:20:54.104405 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4ac8c-a7b9-4c06-9560-26bdb389dacc service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:20:54.135514 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1119/2232] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:20:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:20:58.213860 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600c904-4649-4590-980c-b3317cccc4e3 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:20:58.227422 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600c904-4649-4590-980c-b3317cccc4e3 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:20:58.234091 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550bf829-a449-4e44-a5ca-08e6a0ad8d9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:20:58.235017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550bf829-a449-4e44-a5ca-08e6a0ad8d9a service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:20:58.235142 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550bf829-a449-4e44-a5ca-08e6a0ad8d9a service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:20:58.235339 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550bf829-a449-4e44-a5ca-08e6a0ad8d9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:20:58.244891 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550bf829-a449-4e44-a5ca-08e6a0ad8d9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:20:58.246675 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550bf829-a449-4e44-a5ca-08e6a0ad8d9a service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:20:58.275761 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1114/2233] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:20:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:20:58.780655 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4ac8c-a7b9-4c06-9560-26bdb389dacc service nova] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:20:58.801205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4ac8c-a7b9-4c06-9560-26bdb389dacc service nova] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:20:58.804988 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227b577-2ba2-4916-b7f5-dff47de7d8c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:20:58.805631 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227b577-2ba2-4916-b7f5-dff47de7d8c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:20:58.805908 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227b577-2ba2-4916-b7f5-dff47de7d8c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:20:58.806178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227b577-2ba2-4916-b7f5-dff47de7d8c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:20:58.817737 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8227b577-2ba2-4916-b7f5-dff47de7d8c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:20:58.819194 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8227b577-2ba2-4916-b7f5-dff47de7d8c8 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:20:58.834547 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1120/2234] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:20:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:21:08.272838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=13 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:21:08.826228 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=28 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:21:22.219865 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550bf829-a449-4e44-a5ca-08e6a0ad8d9a service nova] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:21:22.229344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550bf829-a449-4e44-a5ca-08e6a0ad8d9a service nova] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:21:22.233221 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fa989-56e8-4395-9719-223071075f19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:21:22.233948 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fa989-56e8-4395-9719-223071075f19 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:21:22.234234 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fa989-56e8-4395-9719-223071075f19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:21:22.234426 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fa989-56e8-4395-9719-223071075f19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:21:22.243915 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fa989-56e8-4395-9719-223071075f19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:21:22.245338 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fa989-56e8-4395-9719-223071075f19 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:21:22.275048 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1115/2235] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:21:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:21:32.272269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=13 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:21:45.684069 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227b577-2ba2-4916-b7f5-dff47de7d8c8 service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:21:45.695665 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227b577-2ba2-4916-b7f5-dff47de7d8c8 service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:21:45.701374 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ad7dc-c373-4dde-966d-4d0dcd3e024a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:21:45.702093 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ad7dc-c373-4dde-966d-4d0dcd3e024a service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:21:45.702591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ad7dc-c373-4dde-966d-4d0dcd3e024a service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:21:45.703121 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ad7dc-c373-4dde-966d-4d0dcd3e024a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:21:45.714127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ad7dc-c373-4dde-966d-4d0dcd3e024a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:21:45.715730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ad7dc-c373-4dde-966d-4d0dcd3e024a service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:21:45.725369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1121/2236] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:21:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:21:55.721409 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=6 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:23:22.457378 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fa989-56e8-4395-9719-223071075f19 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:23:22.466238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fa989-56e8-4395-9719-223071075f19 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:23:22.470677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d6493-06ea-4c54-b819-d1c2b55521c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:23:22.471358 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d6493-06ea-4c54-b819-d1c2b55521c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:23:22.471648 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d6493-06ea-4c54-b819-d1c2b55521c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:23:22.471917 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d6493-06ea-4c54-b819-d1c2b55521c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:23:22.482452 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d6493-06ea-4c54-b819-d1c2b55521c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:23:22.484486 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d6493-06ea-4c54-b819-d1c2b55521c6 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:23:22.495798 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1116/2237] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:23:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:23:32.490812 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=6 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:23:48.951211 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ad7dc-c373-4dde-966d-4d0dcd3e024a service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:23:48.952609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-083ad7dc-c373-4dde-966d-4d0dcd3e024a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:23:48.968290 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ad7dc-c373-4dde-966d-4d0dcd3e024a service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:23:48.973485 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73e705-f418-45ac-8db6-ae89a1a291d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a30cee766814c8685578819261705ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:23:48.974124 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73e705-f418-45ac-8db6-ae89a1a291d6 service placement] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:23:48.974408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73e705-f418-45ac-8db6-ae89a1a291d6 service placement] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:23:48.974687 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73e705-f418-45ac-8db6-ae89a1a291d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:23:48.983250 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb73e705-f418-45ac-8db6-ae89a1a291d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:23:48.984819 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb73e705-f418-45ac-8db6-ae89a1a291d6 service placement] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:23:48.995451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1122/2238] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:23:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:23:58.991324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=6 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:24:56.271671 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d6493-06ea-4c54-b819-d1c2b55521c6 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:24:56.326371 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d6493-06ea-4c54-b819-d1c2b55521c6 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:24:56.330828 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44ba88-f867-4e10-84d9-17a3c4961707 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:24:56.331496 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44ba88-f867-4e10-84d9-17a3c4961707 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:24:56.331776 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44ba88-f867-4e10-84d9-17a3c4961707 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:24:56.332050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44ba88-f867-4e10-84d9-17a3c4961707 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:24:56.340719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44ba88-f867-4e10-84d9-17a3c4961707 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:24:56.342146 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec44ba88-f867-4e10-84d9-17a3c4961707 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:24:56.369242 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1117/2239] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:24:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:25:06.366486 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=33 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:25:22.898002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73e705-f418-45ac-8db6-ae89a1a291d6 service placement] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:25:22.899651 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-cb73e705-f418-45ac-8db6-ae89a1a291d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:25:22.925010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73e705-f418-45ac-8db6-ae89a1a291d6 service placement] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:25:22.928938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14570b6c-cfdc-4510-8623-87338f8bef74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:25:22.929580 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14570b6c-cfdc-4510-8623-87338f8bef74 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:25:22.929863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14570b6c-cfdc-4510-8623-87338f8bef74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:25:22.930140 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14570b6c-cfdc-4510-8623-87338f8bef74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:25:22.932237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-14570b6c-cfdc-4510-8623-87338f8bef74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:25:23.013894 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14570b6c-cfdc-4510-8623-87338f8bef74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:25:23.015364 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14570b6c-cfdc-4510-8623-87338f8bef74 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:25:23.034541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1123/2240] 10.222.0.27 () {60 vars in 1396 bytes} [Sat Jul 16 15:25:22 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3248 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:25:23.198478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44ba88-f867-4e10-84d9-17a3c4961707 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:25:23.208600 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44ba88-f867-4e10-84d9-17a3c4961707 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:25:23.214506 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29505b7c-ec7f-4955-86f6-7796510e0d02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:25:23.215178 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29505b7c-ec7f-4955-86f6-7796510e0d02 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:25:23.216613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29505b7c-ec7f-4955-86f6-7796510e0d02 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:25:23.216613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29505b7c-ec7f-4955-86f6-7796510e0d02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:25:23.226602 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29505b7c-ec7f-4955-86f6-7796510e0d02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:25:23.228307 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29505b7c-ec7f-4955-86f6-7796510e0d02 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:25:23.251731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14570b6c-cfdc-4510-8623-87338f8bef74 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:25:23.270859 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14570b6c-cfdc-4510-8623-87338f8bef74 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:25:23.278824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc62d14-3a14-48d9-8c53-a9d4384a01b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:25:23.279776 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc62d14-3a14-48d9-8c53-a9d4384a01b9 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:25:23.280576 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc62d14-3a14-48d9-8c53-a9d4384a01b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:25:23.281036 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc62d14-3a14-48d9-8c53-a9d4384a01b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:25:23.283053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1118/2241] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:25:23 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:25:23.395994 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc62d14-3a14-48d9-8c53-a9d4384a01b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:25:23.401551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc62d14-3a14-48d9-8c53-a9d4384a01b9 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:25:23.421998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1124/2242] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:25:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:25:33.267179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=13 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:25:33.419114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=82 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:26:54.274417 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29505b7c-ec7f-4955-86f6-7796510e0d02 service nova] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:26:54.276109 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-29505b7c-ec7f-4955-86f6-7796510e0d02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:26:54.346704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29505b7c-ec7f-4955-86f6-7796510e0d02 service nova] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:26:54.350324 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfcb65-5f76-4618-adc6-987893342ae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:26:54.351005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfcb65-5f76-4618-adc6-987893342ae0 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:26:54.351286 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfcb65-5f76-4618-adc6-987893342ae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:26:54.351559 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dfcb65-5f76-4618-adc6-987893342ae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:26:54.353428 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e7dfcb65-5f76-4618-adc6-987893342ae0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:26:54.364048 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dfcb65-5f76-4618-adc6-987893342ae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:26:54.365547 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dfcb65-5f76-4618-adc6-987893342ae0 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:26:54.378311 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1119/2243] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:26:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:26:56.406347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc62d14-3a14-48d9-8c53-a9d4384a01b9 service nova] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:26:56.408149 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-fcc62d14-3a14-48d9-8c53-a9d4384a01b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:26:56.421021 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc62d14-3a14-48d9-8c53-a9d4384a01b9 service nova] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:26:56.424709 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6b6eb-d20a-4071-8b93-aa0b9a2036b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:26:56.425355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6b6eb-d20a-4071-8b93-aa0b9a2036b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:26:56.425634 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6b6eb-d20a-4071-8b93-aa0b9a2036b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:26:56.425911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6b6eb-d20a-4071-8b93-aa0b9a2036b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:26:56.428039 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-1fc6b6eb-d20a-4071-8b93-aa0b9a2036b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:26:56.501473 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6b6eb-d20a-4071-8b93-aa0b9a2036b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:26:56.502828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc6b6eb-d20a-4071-8b93-aa0b9a2036b1 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:26:56.511870 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1125/2244] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:26:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:26:56.562195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfcb65-5f76-4618-adc6-987893342ae0 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:26:56.571616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dfcb65-5f76-4618-adc6-987893342ae0 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:26:56.575237 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c67100-4fe7-4c33-a303-fd27f1d3a40d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:26:56.575911 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c67100-4fe7-4c33-a303-fd27f1d3a40d service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:26:56.576350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c67100-4fe7-4c33-a303-fd27f1d3a40d service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:26:56.576630 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c67100-4fe7-4c33-a303-fd27f1d3a40d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:26:56.578500 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d1c67100-4fe7-4c33-a303-fd27f1d3a40d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:26:56.650355 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c67100-4fe7-4c33-a303-fd27f1d3a40d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:26:56.651843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c67100-4fe7-4c33-a303-fd27f1d3a40d service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:26:56.661339 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1120/2245] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:26:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:27:06.507746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=33 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:27:06.657225 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=64 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:27:23.431938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6b6eb-d20a-4071-8b93-aa0b9a2036b1 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:27:23.450826 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6b6eb-d20a-4071-8b93-aa0b9a2036b1 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:27:23.455145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1da06-6f93-49b2-abc9-2463fe58695c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:27:23.455795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1da06-6f93-49b2-abc9-2463fe58695c service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:27:23.456070 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1da06-6f93-49b2-abc9-2463fe58695c service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:27:23.456339 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1da06-6f93-49b2-abc9-2463fe58695c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:27:23.458245 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-f1d1da06-6f93-49b2-abc9-2463fe58695c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:27:23.574183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1da06-6f93-49b2-abc9-2463fe58695c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:27:23.575715 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1da06-6f93-49b2-abc9-2463fe58695c service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:27:23.588932 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1126/2246] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:27:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:27:33.583366 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=56 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:28:54.961701 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c67100-4fe7-4c33-a303-fd27f1d3a40d service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:28:55.064383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c67100-4fe7-4c33-a303-fd27f1d3a40d service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:28:55.069763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873b531-e09e-405c-84c0-17e9f2be6fd4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a30cee766814c8685578819261705ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:28:55.070457 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873b531-e09e-405c-84c0-17e9f2be6fd4 service placement] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:28:55.070737 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873b531-e09e-405c-84c0-17e9f2be6fd4 service placement] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:28:55.071637 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873b531-e09e-405c-84c0-17e9f2be6fd4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:28:55.073563 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-0873b531-e09e-405c-84c0-17e9f2be6fd4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:28:55.098610 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873b531-e09e-405c-84c0-17e9f2be6fd4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:28:55.100264 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873b531-e09e-405c-84c0-17e9f2be6fd4 service placement] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:28:55.110860 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1121/2247] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:28:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:29:05.106665 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=52 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:29:23.920910 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1da06-6f93-49b2-abc9-2463fe58695c service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:29:23.932174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873b531-e09e-405c-84c0-17e9f2be6fd4 service placement] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:29:23.932174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-0873b531-e09e-405c-84c0-17e9f2be6fd4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:29:23.939911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1da06-6f93-49b2-abc9-2463fe58695c service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:29:23.943687 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e2b84-5483-4eb9-b6d8-4e908fb99835 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:29:23.944365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e2b84-5483-4eb9-b6d8-4e908fb99835 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:29:23.944649 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e2b84-5483-4eb9-b6d8-4e908fb99835 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:29:23.944926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e2b84-5483-4eb9-b6d8-4e908fb99835 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:29:23.946854 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ae6e2b84-5483-4eb9-b6d8-4e908fb99835 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:29:23.961260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e2b84-5483-4eb9-b6d8-4e908fb99835 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:29:23.962570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873b531-e09e-405c-84c0-17e9f2be6fd4 service placement] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:29:23.962700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e2b84-5483-4eb9-b6d8-4e908fb99835 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:29:23.968470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bab920-1533-4a0d-a164-982b4eae6259 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:29:23.970567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bab920-1533-4a0d-a164-982b4eae6259 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:29:23.970862 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bab920-1533-4a0d-a164-982b4eae6259 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:29:23.971177 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bab920-1533-4a0d-a164-982b4eae6259 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:29:23.973090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-59bab920-1533-4a0d-a164-982b4eae6259 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:29:23.974752 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1127/2248] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:29:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:29:24.053327 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bab920-1533-4a0d-a164-982b4eae6259 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:29:24.055055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bab920-1533-4a0d-a164-982b4eae6259 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:29:24.066505 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1122/2249] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:29:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:29:24.527034 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e2b84-5483-4eb9-b6d8-4e908fb99835 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:29:24.539147 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e2b84-5483-4eb9-b6d8-4e908fb99835 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:29:24.545515 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9549a7-1280-4c07-bfeb-5a079b66e6c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:29:24.546173 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9549a7-1280-4c07-bfeb-5a079b66e6c3 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:29:24.546473 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9549a7-1280-4c07-bfeb-5a079b66e6c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:29:24.546776 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9549a7-1280-4c07-bfeb-5a079b66e6c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:29:24.548786 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-9e9549a7-1280-4c07-bfeb-5a079b66e6c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:29:24.569206 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9549a7-1280-4c07-bfeb-5a079b66e6c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:29:24.569206 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9549a7-1280-4c07-bfeb-5a079b66e6c3 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:29:24.575426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1128/2250] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:29:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:29:34.061117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=40 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:29:34.570703 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=16 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:30:57.033260 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bab920-1533-4a0d-a164-982b4eae6259 service nova] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:30:57.042843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bab920-1533-4a0d-a164-982b4eae6259 service nova] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:30:57.048188 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e362b-d2ac-4ba0-9879-c3f164a5f3bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:30:57.048188 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e362b-d2ac-4ba0-9879-c3f164a5f3bf service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:30:57.048188 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e362b-d2ac-4ba0-9879-c3f164a5f3bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:30:57.048188 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e362b-d2ac-4ba0-9879-c3f164a5f3bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:30:57.050297 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6f0e362b-d2ac-4ba0-9879-c3f164a5f3bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:30:57.125074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e362b-d2ac-4ba0-9879-c3f164a5f3bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:30:57.126512 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e362b-d2ac-4ba0-9879-c3f164a5f3bf service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:30:57.136558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1123/2251] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:30:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:30:57.188707 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9549a7-1280-4c07-bfeb-5a079b66e6c3 service nova] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:30:57.198180 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9549a7-1280-4c07-bfeb-5a079b66e6c3 service nova] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:30:57.201742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0856a-97ef-41d4-a49f-9ec8fa93d866 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:30:57.202391 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0856a-97ef-41d4-a49f-9ec8fa93d866 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:30:57.202673 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0856a-97ef-41d4-a49f-9ec8fa93d866 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:30:57.202979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0856a-97ef-41d4-a49f-9ec8fa93d866 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:30:57.212372 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0856a-97ef-41d4-a49f-9ec8fa93d866 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:30:57.213846 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0856a-97ef-41d4-a49f-9ec8fa93d866 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:30:57.224126 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1129/2252] 10.222.0.27 () {60 vars in 1396 bytes} [Sat Jul 16 15:30:57 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3248 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:31:07.132373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=33 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:31:07.219142 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=6 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:31:24.450094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e362b-d2ac-4ba0-9879-c3f164a5f3bf service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:31:24.464655 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e362b-d2ac-4ba0-9879-c3f164a5f3bf service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:31:24.470424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc65c3d-6e14-4570-888b-18c77c37e1b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:31:24.471426 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc65c3d-6e14-4570-888b-18c77c37e1b6 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:31:24.471711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc65c3d-6e14-4570-888b-18c77c37e1b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:31:24.471985 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc65c3d-6e14-4570-888b-18c77c37e1b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:31:24.473945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9cc65c3d-6e14-4570-888b-18c77c37e1b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:31:24.555933 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc65c3d-6e14-4570-888b-18c77c37e1b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:31:24.557553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc65c3d-6e14-4570-888b-18c77c37e1b6 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:31:24.567132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1124/2253] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:31:24 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:31:24.581549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0856a-97ef-41d4-a49f-9ec8fa93d866 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:31:24.583211 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5ba0856a-97ef-41d4-a49f-9ec8fa93d866 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:31:24.615942 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0856a-97ef-41d4-a49f-9ec8fa93d866 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:31:24.628129 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a01fba-117f-4e78-8b62-fc5e987415a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:31:24.628804 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a01fba-117f-4e78-8b62-fc5e987415a9 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:31:24.629091 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a01fba-117f-4e78-8b62-fc5e987415a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:31:24.629369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a01fba-117f-4e78-8b62-fc5e987415a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:31:24.632695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-13a01fba-117f-4e78-8b62-fc5e987415a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:31:24.660989 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a01fba-117f-4e78-8b62-fc5e987415a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:31:24.663574 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a01fba-117f-4e78-8b62-fc5e987415a9 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:31:24.674013 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1130/2254] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:31:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:31:25.844927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f6eb9-1589-464c-adc1-57ced9591ef0 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:31:25.845501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f6eb9-1589-464c-adc1-57ced9591ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:31:25.845501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f6eb9-1589-464c-adc1-57ced9591ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:31:25.903931 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-8d9f6eb9-1589-464c-adc1-57ced9591ef0 None None] MFA Rules not processed for user `e97b2376802c4e3f936eaa3d33b5b194`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:31:25.905282 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-8d9f6eb9-1589-464c-adc1-57ced9591ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:31:25.915167 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1125/2255] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:31:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:31:26.692904 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a01fba-117f-4e78-8b62-fc5e987415a9 service nova] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:31:26.694902 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-13a01fba-117f-4e78-8b62-fc5e987415a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:31:26.718985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a01fba-117f-4e78-8b62-fc5e987415a9 service nova] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:31:26.722864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918c5cd-4ee9-4d8c-b25c-f03117709975 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:31:26.723541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918c5cd-4ee9-4d8c-b25c-f03117709975 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:31:26.723824 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918c5cd-4ee9-4d8c-b25c-f03117709975 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:31:26.724098 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918c5cd-4ee9-4d8c-b25c-f03117709975 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:31:26.726492 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-9918c5cd-4ee9-4d8c-b25c-f03117709975 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:31:26.737455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9918c5cd-4ee9-4d8c-b25c-f03117709975 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:31:26.738951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9918c5cd-4ee9-4d8c-b25c-f03117709975 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:31:26.750791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1131/2256] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:31:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:31:27.525545 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f6eb9-1589-464c-adc1-57ced9591ef0 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:31:27.546730 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f6eb9-1589-464c-adc1-57ced9591ef0 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:31:27.554665 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e9a87-088d-4fc3-ae8f-4c839565f057 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:31:27.555682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e9a87-088d-4fc3-ae8f-4c839565f057 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:31:27.556172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e9a87-088d-4fc3-ae8f-4c839565f057 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:31:27.557803 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e9a87-088d-4fc3-ae8f-4c839565f057 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:31:27.558679 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d38e9a87-088d-4fc3-ae8f-4c839565f057 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:31:27.574553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e9a87-088d-4fc3-ae8f-4c839565f057 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:31:27.576382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e9a87-088d-4fc3-ae8f-4c839565f057 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:31:27.591144 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1126/2257] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:31:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:31:29.590119 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658d3c3-af93-496d-9acf-e000240d672b None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:31:29.593008 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658d3c3-af93-496d-9acf-e000240d672b None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:31:29.593008 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658d3c3-af93-496d-9acf-e000240d672b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:31:29.627333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-7658d3c3-af93-496d-9acf-e000240d672b None None] MFA Rules not processed for user `096fd1b2e2b4482da1026d3fb7e192c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:31:29.628545 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7658d3c3-af93-496d-9acf-e000240d672b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:31:29.644844 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1132/2258] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:31:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:31:29.967933 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e9a87-088d-4fc3-ae8f-4c839565f057 service nova] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:31:29.978184 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e9a87-088d-4fc3-ae8f-4c839565f057 service nova] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:31:29.981924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab48a3-4608-428f-9b87-d7ae26bf78bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:31:29.982564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab48a3-4608-428f-9b87-d7ae26bf78bb service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:31:29.982847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab48a3-4608-428f-9b87-d7ae26bf78bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:31:29.983178 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab48a3-4608-428f-9b87-d7ae26bf78bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:31:29.984950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7dab48a3-4608-428f-9b87-d7ae26bf78bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:31:29.994818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab48a3-4608-428f-9b87-d7ae26bf78bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:31:29.997566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dab48a3-4608-428f-9b87-d7ae26bf78bb service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:31:30.004329 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1127/2259] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:31:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:31:30.409085 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7658d3c3-af93-496d-9acf-e000240d672b None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:31:30.422300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7658d3c3-af93-496d-9acf-e000240d672b None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:31:30.429345 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389599c8-5b2d-40e2-b724-754393f33dce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a30cee766814c8685578819261705ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:31:30.430115 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389599c8-5b2d-40e2-b724-754393f33dce service placement] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:31:30.430490 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389599c8-5b2d-40e2-b724-754393f33dce service placement] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:31:30.431010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389599c8-5b2d-40e2-b724-754393f33dce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:31:30.433614 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-389599c8-5b2d-40e2-b724-754393f33dce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:31:30.446340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389599c8-5b2d-40e2-b724-754393f33dce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:31:30.448970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389599c8-5b2d-40e2-b724-754393f33dce service placement] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:31:30.458739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1133/2260] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:31:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:31:40.000691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=51 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:31:40.210350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab48a3-4608-428f-9b87-d7ae26bf78bb service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:31:40.226330 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dab48a3-4608-428f-9b87-d7ae26bf78bb service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:31:40.231881 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04d965-194f-4181-869f-9e6c4e548ae7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:31:40.232521 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04d965-194f-4181-869f-9e6c4e548ae7 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:31:40.232800 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04d965-194f-4181-869f-9e6c4e548ae7 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:31:40.233071 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04d965-194f-4181-869f-9e6c4e548ae7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:31:40.234955 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ca04d965-194f-4181-869f-9e6c4e548ae7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:31:40.248001 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04d965-194f-4181-869f-9e6c4e548ae7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:31:40.249572 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04d965-194f-4181-869f-9e6c4e548ae7 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:31:40.260831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1128/2261] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:31:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:31:40.454426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=46 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:31:50.256223 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=6 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:32:01.733612 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389599c8-5b2d-40e2-b724-754393f33dce service placement] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:32:01.744022 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389599c8-5b2d-40e2-b724-754393f33dce service placement] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:32:01.747619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f45eee-0083-4e11-9e19-456e08edf005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:32:01.748257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f45eee-0083-4e11-9e19-456e08edf005 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:32:01.748541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f45eee-0083-4e11-9e19-456e08edf005 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:32:01.748816 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f45eee-0083-4e11-9e19-456e08edf005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:32:01.757082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f45eee-0083-4e11-9e19-456e08edf005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:32:01.758676 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f45eee-0083-4e11-9e19-456e08edf005 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:32:01.767811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1134/2262] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:32:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:32:11.764697 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=6 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:32:57.396543 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04d965-194f-4181-869f-9e6c4e548ae7 service nova] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:32:57.406542 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04d965-194f-4181-869f-9e6c4e548ae7 service nova] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:32:57.410090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8819c2c-5445-49d6-8c76-3f11b8311aa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:32:57.410735 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8819c2c-5445-49d6-8c76-3f11b8311aa6 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:32:57.411057 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8819c2c-5445-49d6-8c76-3f11b8311aa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:32:57.411332 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8819c2c-5445-49d6-8c76-3f11b8311aa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:32:57.420549 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8819c2c-5445-49d6-8c76-3f11b8311aa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:32:57.421977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8819c2c-5445-49d6-8c76-3f11b8311aa6 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:32:57.449007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1129/2263] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:32:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:32:57.496588 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f45eee-0083-4e11-9e19-456e08edf005 service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:32:57.506013 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f45eee-0083-4e11-9e19-456e08edf005 service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:32:57.509607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d8460-c2c5-4be6-b737-6b2908ac25f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:32:57.510413 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d8460-c2c5-4be6-b737-6b2908ac25f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:32:57.510968 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d8460-c2c5-4be6-b737-6b2908ac25f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:32:57.511300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d8460-c2c5-4be6-b737-6b2908ac25f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:32:57.520371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d8460-c2c5-4be6-b737-6b2908ac25f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:32:57.521790 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d8460-c2c5-4be6-b737-6b2908ac25f5 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:32:57.551484 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1135/2264] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:32:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:33:07.446276 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=13 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:33:07.548895 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=13 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:33:58.984309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8819c2c-5445-49d6-8c76-3f11b8311aa6 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:33:58.985923 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e8819c2c-5445-49d6-8c76-3f11b8311aa6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:33:59.001032 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8819c2c-5445-49d6-8c76-3f11b8311aa6 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:33:59.006029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855f689-5d76-4c37-b06e-8c3373fb0e34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a30cee766814c8685578819261705ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:33:59.006689 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855f689-5d76-4c37-b06e-8c3373fb0e34 service placement] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:33:59.007012 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855f689-5d76-4c37-b06e-8c3373fb0e34 service placement] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:33:59.007294 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855f689-5d76-4c37-b06e-8c3373fb0e34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:33:59.017648 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855f689-5d76-4c37-b06e-8c3373fb0e34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:33:59.019409 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855f689-5d76-4c37-b06e-8c3373fb0e34 service placement] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:33:59.032241 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1130/2265] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:33:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:34:09.026083 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=6 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:34:23.459460 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d8460-c2c5-4be6-b737-6b2908ac25f5 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:34:23.468576 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d8460-c2c5-4be6-b737-6b2908ac25f5 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:34:23.472889 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627b57e-730e-495d-8242-dbaa9fde69af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:34:23.473517 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627b57e-730e-495d-8242-dbaa9fde69af service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:34:23.473797 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627b57e-730e-495d-8242-dbaa9fde69af service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:34:23.474071 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627b57e-730e-495d-8242-dbaa9fde69af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:34:23.484165 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627b57e-730e-495d-8242-dbaa9fde69af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:34:23.485721 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627b57e-730e-495d-8242-dbaa9fde69af service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:34:23.496431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1136/2266] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:34:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:34:33.491694 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=6 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:34:58.279117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855f689-5d76-4c37-b06e-8c3373fb0e34 service placement] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:34:58.334620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855f689-5d76-4c37-b06e-8c3373fb0e34 service placement] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:34:58.338247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f254a-24ee-4a50-9928-f4c33ab388e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:34:58.338916 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f254a-24ee-4a50-9928-f4c33ab388e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:34:58.339214 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f254a-24ee-4a50-9928-f4c33ab388e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:34:58.339493 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f254a-24ee-4a50-9928-f4c33ab388e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:34:58.348219 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f254a-24ee-4a50-9928-f4c33ab388e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:34:58.349644 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f254a-24ee-4a50-9928-f4c33ab388e5 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:34:58.358465 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1131/2267] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:34:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:35:08.354422 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=26 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:35:57.418307 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627b57e-730e-495d-8242-dbaa9fde69af service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:35:57.429724 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627b57e-730e-495d-8242-dbaa9fde69af service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:35:57.435201 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470cdcbe-217c-4a6d-921a-4a8a348786e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:35:57.435832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cdcbe-217c-4a6d-921a-4a8a348786e7 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:35:57.436112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cdcbe-217c-4a6d-921a-4a8a348786e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:35:57.436384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cdcbe-217c-4a6d-921a-4a8a348786e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:35:57.444737 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470cdcbe-217c-4a6d-921a-4a8a348786e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:35:57.446297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470cdcbe-217c-4a6d-921a-4a8a348786e7 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:35:57.473619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1137/2268] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:35:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:36:07.471018 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=13 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:36:25.426160 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f254a-24ee-4a50-9928-f4c33ab388e5 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:36:25.438531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f254a-24ee-4a50-9928-f4c33ab388e5 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:36:25.443977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea029983-fa61-4a4d-b114-174ecba359cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:36:25.444641 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea029983-fa61-4a4d-b114-174ecba359cb service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:36:25.444922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea029983-fa61-4a4d-b114-174ecba359cb service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:36:25.445197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea029983-fa61-4a4d-b114-174ecba359cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:36:25.453186 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea029983-fa61-4a4d-b114-174ecba359cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:36:25.463602 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea029983-fa61-4a4d-b114-174ecba359cb service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:36:25.482107 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1132/2269] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:36:25 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:36:35.479297 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=13 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:36:58.446272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470cdcbe-217c-4a6d-921a-4a8a348786e7 service nova] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:36:58.499599 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470cdcbe-217c-4a6d-921a-4a8a348786e7 service nova] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:36:58.503406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4732f8-4b05-4574-bb38-cec2714a733e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:36:58.504041 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4732f8-4b05-4574-bb38-cec2714a733e service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:36:58.504329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4732f8-4b05-4574-bb38-cec2714a733e service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:36:58.504591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4732f8-4b05-4574-bb38-cec2714a733e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:36:58.514659 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4732f8-4b05-4574-bb38-cec2714a733e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:36:58.516160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4732f8-4b05-4574-bb38-cec2714a733e service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:36:58.527529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1138/2270] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:36:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:36:58.829234 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea029983-fa61-4a4d-b114-174ecba359cb service nova] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:36:58.831382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ea029983-fa61-4a4d-b114-174ecba359cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:36:58.855634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea029983-fa61-4a4d-b114-174ecba359cb service nova] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:36:58.859181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eac609-6c60-4590-9f8c-5ebd79bb7faf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:36:58.859815 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eac609-6c60-4590-9f8c-5ebd79bb7faf service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:36:58.860094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eac609-6c60-4590-9f8c-5ebd79bb7faf service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:36:58.860367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eac609-6c60-4590-9f8c-5ebd79bb7faf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:36:58.862302 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-50eac609-6c60-4590-9f8c-5ebd79bb7faf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:36:58.943370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eac609-6c60-4590-9f8c-5ebd79bb7faf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:36:58.944808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eac609-6c60-4590-9f8c-5ebd79bb7faf service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:36:58.963305 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1133/2271] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:36:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:36:59.011973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4732f8-4b05-4574-bb38-cec2714a733e service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:36:59.021703 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4732f8-4b05-4574-bb38-cec2714a733e service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:36:59.025380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6833e8-4630-4d3f-b269-06a624a90710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:36:59.026072 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6833e8-4630-4d3f-b269-06a624a90710 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:36:59.026351 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6833e8-4630-4d3f-b269-06a624a90710 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:36:59.026625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6833e8-4630-4d3f-b269-06a624a90710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:36:59.090597 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6833e8-4630-4d3f-b269-06a624a90710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:36:59.092023 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6833e8-4630-4d3f-b269-06a624a90710 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:36:59.109761 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1139/2272] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:36:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:37:08.960663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=46 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:37:09.107149 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=62 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:37:10.657108 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eac609-6c60-4590-9f8c-5ebd79bb7faf service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:37:10.666427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eac609-6c60-4590-9f8c-5ebd79bb7faf service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:37:10.670047 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16590eea-3388-49c1-b177-4fe986fd3372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:37:10.670690 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16590eea-3388-49c1-b177-4fe986fd3372 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:37:10.671004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16590eea-3388-49c1-b177-4fe986fd3372 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:37:10.671541 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16590eea-3388-49c1-b177-4fe986fd3372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:37:10.673437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-16590eea-3388-49c1-b177-4fe986fd3372 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:37:10.696823 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16590eea-3388-49c1-b177-4fe986fd3372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:37:10.698256 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16590eea-3388-49c1-b177-4fe986fd3372 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:37:10.706406 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1134/2273] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:37:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:37:20.702734 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=11 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:37:28.640334 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6833e8-4630-4d3f-b269-06a624a90710 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:37:28.650651 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6833e8-4630-4d3f-b269-06a624a90710 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:37:28.658063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428d195-0feb-490a-a2cb-c7e19c8acdb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:37:28.658696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428d195-0feb-490a-a2cb-c7e19c8acdb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:37:28.660014 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428d195-0feb-490a-a2cb-c7e19c8acdb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:37:28.668750 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428d195-0feb-490a-a2cb-c7e19c8acdb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:37:28.668750 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a428d195-0feb-490a-a2cb-c7e19c8acdb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:37:28.779681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428d195-0feb-490a-a2cb-c7e19c8acdb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:37:28.781129 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428d195-0feb-490a-a2cb-c7e19c8acdb6 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:37:28.803287 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1140/2274] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:37:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:37:31.068015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16590eea-3388-49c1-b177-4fe986fd3372 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:37:31.071273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-16590eea-3388-49c1-b177-4fe986fd3372 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:37:31.115692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16590eea-3388-49c1-b177-4fe986fd3372 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:37:31.120109 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00df04-1735-45b5-95c9-eb18fa2cc1d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:37:31.120984 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00df04-1735-45b5-95c9-eb18fa2cc1d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:37:31.121426 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00df04-1735-45b5-95c9-eb18fa2cc1d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:37:31.122364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00df04-1735-45b5-95c9-eb18fa2cc1d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:37:31.125260 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4f00df04-1735-45b5-95c9-eb18fa2cc1d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:37:31.139281 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00df04-1735-45b5-95c9-eb18fa2cc1d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:37:31.141543 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00df04-1735-45b5-95c9-eb18fa2cc1d8 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:37:31.153455 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1135/2275] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:37:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:37:33.803884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428d195-0feb-490a-a2cb-c7e19c8acdb6 service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:37:33.818432 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428d195-0feb-490a-a2cb-c7e19c8acdb6 service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:37:33.824410 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e5412-a1fa-47e2-8364-811b726ed384 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a30cee766814c8685578819261705ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:37:33.825142 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e5412-a1fa-47e2-8364-811b726ed384 service placement] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:37:33.825553 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e5412-a1fa-47e2-8364-811b726ed384 service placement] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:37:33.825902 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e5412-a1fa-47e2-8364-811b726ed384 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:37:33.828900 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-cb1e5412-a1fa-47e2-8364-811b726ed384 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:37:33.843217 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e5412-a1fa-47e2-8364-811b726ed384 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:37:33.844929 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e5412-a1fa-47e2-8364-811b726ed384 service placement] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:37:33.864047 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1141/2276] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:37:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:37:39.582230 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00df04-1735-45b5-95c9-eb18fa2cc1d8 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:37:39.598106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00df04-1735-45b5-95c9-eb18fa2cc1d8 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:37:39.607063 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac6a45-7c45-4869-84c3-d18b409b0d36 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a30cee766814c8685578819261705ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:37:39.608688 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac6a45-7c45-4869-84c3-d18b409b0d36 service placement] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:37:39.609269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac6a45-7c45-4869-84c3-d18b409b0d36 service placement] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:37:39.610090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac6a45-7c45-4869-84c3-d18b409b0d36 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:37:39.613372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1aac6a45-7c45-4869-84c3-d18b409b0d36 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:37:39.629410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac6a45-7c45-4869-84c3-d18b409b0d36 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:37:39.631801 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac6a45-7c45-4869-84c3-d18b409b0d36 service placement] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:37:39.645318 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1136/2277] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:37:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:37:39.796982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e5412-a1fa-47e2-8364-811b726ed384 service placement] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:37:39.798783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-cb1e5412-a1fa-47e2-8364-811b726ed384 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:37:39.824340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e5412-a1fa-47e2-8364-811b726ed384 service placement] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:37:39.832035 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db63631-69db-4078-8a43-182d90fc5e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:37:39.833078 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db63631-69db-4078-8a43-182d90fc5e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:37:39.833557 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db63631-69db-4078-8a43-182d90fc5e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:37:39.833962 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db63631-69db-4078-8a43-182d90fc5e30 admin admin] PATH_INFO: `/v3/users/ff8ae9f8a9db47fdbc5e63fffb2f78b1` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:37:39.835425 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db63631-69db-4078-8a43-182d90fc5e30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff8ae9f8a9db47fdbc5e63fffb2f78b1)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:37:39.838120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db63631-69db-4078-8a43-182d90fc5e30 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:37:40.091356 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-3db63631-69db-4078-8a43-182d90fc5e30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff8ae9f8a9db47fdbc5e63fffb2f78b1'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:40.184757 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-3db63631-69db-4078-8a43-182d90fc5e30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff8ae9f8a9db47fdbc5e63fffb2f78b1'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:40.200984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-3db63631-69db-4078-8a43-182d90fc5e30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff8ae9f8a9db47fdbc5e63fffb2f78b1'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:40.217710 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-3db63631-69db-4078-8a43-182d90fc5e30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff8ae9f8a9db47fdbc5e63fffb2f78b1'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:40.218573 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-3db63631-69db-4078-8a43-182d90fc5e30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff8ae9f8a9db47fdbc5e63fffb2f78b1'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:40.233492 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1142/2278] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:37:39 2022] DELETE /identity/v3/users/ff8ae9f8a9db47fdbc5e63fffb2f78b1 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:37:40.528492 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac6a45-7c45-4869-84c3-d18b409b0d36 service placement] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:37:40.551264 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac6a45-7c45-4869-84c3-d18b409b0d36 service placement] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:37:40.725683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2839c3b-3234-4082-8116-be1f7a25d58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:37:40.726573 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2839c3b-3234-4082-8116-be1f7a25d58a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:37:40.726932 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2839c3b-3234-4082-8116-be1f7a25d58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:37:40.727311 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2839c3b-3234-4082-8116-be1f7a25d58a admin admin] PATH_INFO: `/v3/projects/d771120895ed4423a2dbf25c1c9b2732` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:37:40.728635 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2839c3b-3234-4082-8116-be1f7a25d58a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d771120895ed4423a2dbf25c1c9b2732)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:37:40.731071 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2839c3b-3234-4082-8116-be1f7a25d58a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:37:40.912935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f2839c3b-3234-4082-8116-be1f7a25d58a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd771120895ed4423a2dbf25c1c9b2732', 'request_id': 'req-f2839c3b-3234-4082-8116-be1f7a25d58a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:40.935776 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f2839c3b-3234-4082-8116-be1f7a25d58a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd771120895ed4423a2dbf25c1c9b2732', 'request_id': 'req-f2839c3b-3234-4082-8116-be1f7a25d58a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:40.946491 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f2839c3b-3234-4082-8116-be1f7a25d58a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd771120895ed4423a2dbf25c1c9b2732', 'request_id': 'req-f2839c3b-3234-4082-8116-be1f7a25d58a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:41.059321 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f2839c3b-3234-4082-8116-be1f7a25d58a admin admin] The token cache is being invalidate because project d771120895ed4423a2dbf25c1c9b2732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:37:41.060831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f2839c3b-3234-4082-8116-be1f7a25d58a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:41.061806 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f2839c3b-3234-4082-8116-be1f7a25d58a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:41.064830 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1137/2279] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:37:40 2022] DELETE /identity/v3/projects/d771120895ed4423a2dbf25c1c9b2732 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:37:41.068998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db63631-69db-4078-8a43-182d90fc5e30 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:37:41.070500 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3db63631-69db-4078-8a43-182d90fc5e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:37:41.102250 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db63631-69db-4078-8a43-182d90fc5e30 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:37:41.110302 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:37:41.134207 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2839c3b-3234-4082-8116-be1f7a25d58a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:37:41.145254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f2839c3b-3234-4082-8116-be1f7a25d58a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:37:41.188430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd60dc1-4f4e-4c97-92e9-a396c0fe9f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:37:41.189505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd60dc1-4f4e-4c97-92e9-a396c0fe9f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:37:41.190343 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd60dc1-4f4e-4c97-92e9-a396c0fe9f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:37:41.191019 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd60dc1-4f4e-4c97-92e9-a396c0fe9f91 admin admin] PATH_INFO: `/v3/users/e41aae123ec64a12a614635f23368cd2` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:37:41.207570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:37:41.210036 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd60dc1-4f4e-4c97-92e9-a396c0fe9f91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e41aae123ec64a12a614635f23368cd2)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:37:41.210937 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd60dc1-4f4e-4c97-92e9-a396c0fe9f91 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:37:41.229926 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2839c3b-3234-4082-8116-be1f7a25d58a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:37:41.235879 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b438ae-c155-4099-bfd0-d3efb666a1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:37:41.236998 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b438ae-c155-4099-bfd0-d3efb666a1da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:37:41.237421 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b438ae-c155-4099-bfd0-d3efb666a1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:37:41.237795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b438ae-c155-4099-bfd0-d3efb666a1da admin admin] PATH_INFO: `/v3/users/e97b2376802c4e3f936eaa3d33b5b194` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:37:41.239293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b438ae-c155-4099-bfd0-d3efb666a1da admin admin] RBAC: Authorizing `identity:delete_user(user_id=e97b2376802c4e3f936eaa3d33b5b194)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:37:41.241527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b438ae-c155-4099-bfd0-d3efb666a1da admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:37:41.457042 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-4bd60dc1-4f4e-4c97-92e9-a396c0fe9f91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e41aae123ec64a12a614635f23368cd2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:41.485392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-61b438ae-c155-4099-bfd0-d3efb666a1da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e97b2376802c4e3f936eaa3d33b5b194'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:41.522847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-4bd60dc1-4f4e-4c97-92e9-a396c0fe9f91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e41aae123ec64a12a614635f23368cd2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:41.530284 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-4bd60dc1-4f4e-4c97-92e9-a396c0fe9f91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e41aae123ec64a12a614635f23368cd2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:41.542968 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-4bd60dc1-4f4e-4c97-92e9-a396c0fe9f91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e41aae123ec64a12a614635f23368cd2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:41.543557 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-4bd60dc1-4f4e-4c97-92e9-a396c0fe9f91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e41aae123ec64a12a614635f23368cd2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:41.550842 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1143/2280] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:37:41 2022] DELETE /identity/v3/users/e41aae123ec64a12a614635f23368cd2 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:37:41.571159 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-61b438ae-c155-4099-bfd0-d3efb666a1da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e97b2376802c4e3f936eaa3d33b5b194'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:41.571773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-61b438ae-c155-4099-bfd0-d3efb666a1da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e97b2376802c4e3f936eaa3d33b5b194'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:41.572372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-61b438ae-c155-4099-bfd0-d3efb666a1da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e97b2376802c4e3f936eaa3d33b5b194'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:41.580227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-61b438ae-c155-4099-bfd0-d3efb666a1da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e97b2376802c4e3f936eaa3d33b5b194'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:41.599508 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1138/2281] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:37:41 2022] DELETE /identity/v3/users/e97b2376802c4e3f936eaa3d33b5b194 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:37:41.812369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd60dc1-4f4e-4c97-92e9-a396c0fe9f91 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:37:41.813861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-4bd60dc1-4f4e-4c97-92e9-a396c0fe9f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:37:41.838639 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b438ae-c155-4099-bfd0-d3efb666a1da admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:37:41.854470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-61b438ae-c155-4099-bfd0-d3efb666a1da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:37:41.914606 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd60dc1-4f4e-4c97-92e9-a396c0fe9f91 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:37:41.944762 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:37:41.955475 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b438ae-c155-4099-bfd0-d3efb666a1da admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:37:41.961574 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f67d0-8c3d-4161-b19f-3d3a2f965905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:37:41.962549 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f67d0-8c3d-4161-b19f-3d3a2f965905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:37:41.962824 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f67d0-8c3d-4161-b19f-3d3a2f965905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:37:41.963168 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f67d0-8c3d-4161-b19f-3d3a2f965905 admin admin] PATH_INFO: `/v3/projects/979ae4d7222a4afd84ecd7fb7dc3552f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:37:41.964384 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f67d0-8c3d-4161-b19f-3d3a2f965905 admin admin] RBAC: Authorizing `identity:delete_project(project_id=979ae4d7222a4afd84ecd7fb7dc3552f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:37:41.966285 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f67d0-8c3d-4161-b19f-3d3a2f965905 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:37:41.988106 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597504a-a158-4406-a6fb-cb7f27493278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:37:41.988978 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597504a-a158-4406-a6fb-cb7f27493278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:37:41.989259 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597504a-a158-4406-a6fb-cb7f27493278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:37:41.989661 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597504a-a158-4406-a6fb-cb7f27493278 admin admin] PATH_INFO: `/v3/projects/9d3ee05d7001436a83c3dc5a7a2c3628` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:37:42.001203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597504a-a158-4406-a6fb-cb7f27493278 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d3ee05d7001436a83c3dc5a7a2c3628)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:37:42.003983 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597504a-a158-4406-a6fb-cb7f27493278 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:37:42.109284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-227f67d0-8c3d-4161-b19f-3d3a2f965905 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '979ae4d7222a4afd84ecd7fb7dc3552f', 'request_id': 'req-227f67d0-8c3d-4161-b19f-3d3a2f965905'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:42.114963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-227f67d0-8c3d-4161-b19f-3d3a2f965905 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '979ae4d7222a4afd84ecd7fb7dc3552f', 'request_id': 'req-227f67d0-8c3d-4161-b19f-3d3a2f965905'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:42.121164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-227f67d0-8c3d-4161-b19f-3d3a2f965905 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '979ae4d7222a4afd84ecd7fb7dc3552f', 'request_id': 'req-227f67d0-8c3d-4161-b19f-3d3a2f965905'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:42.125188 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:37:42.142186 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7597504a-a158-4406-a6fb-cb7f27493278 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d3ee05d7001436a83c3dc5a7a2c3628', 'request_id': 'req-7597504a-a158-4406-a6fb-cb7f27493278'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:42.148324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7597504a-a158-4406-a6fb-cb7f27493278 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d3ee05d7001436a83c3dc5a7a2c3628', 'request_id': 'req-7597504a-a158-4406-a6fb-cb7f27493278'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:42.155056 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7597504a-a158-4406-a6fb-cb7f27493278 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d3ee05d7001436a83c3dc5a7a2c3628', 'request_id': 'req-7597504a-a158-4406-a6fb-cb7f27493278'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:42.233745 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-227f67d0-8c3d-4161-b19f-3d3a2f965905 admin admin] The token cache is being invalidate because project 979ae4d7222a4afd84ecd7fb7dc3552f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:37:42.234038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-227f67d0-8c3d-4161-b19f-3d3a2f965905 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:42.234660 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-227f67d0-8c3d-4161-b19f-3d3a2f965905 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:42.236261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1139/2282] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:37:41 2022] DELETE /identity/v3/projects/979ae4d7222a4afd84ecd7fb7dc3552f => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:37:42.240277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f67d0-8c3d-4161-b19f-3d3a2f965905 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:37:42.241756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-227f67d0-8c3d-4161-b19f-3d3a2f965905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:37:42.287244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7597504a-a158-4406-a6fb-cb7f27493278 admin admin] The token cache is being invalidate because project 9d3ee05d7001436a83c3dc5a7a2c3628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:37:42.287588 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7597504a-a158-4406-a6fb-cb7f27493278 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:42.288133 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7597504a-a158-4406-a6fb-cb7f27493278 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:42.289957 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1144/2283] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:37:41 2022] DELETE /identity/v3/projects/9d3ee05d7001436a83c3dc5a7a2c3628 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:37:42.318146 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f67d0-8c3d-4161-b19f-3d3a2f965905 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:37:42.386065 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4614f39-e56a-47a0-921b-47d781557007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:37:42.386879 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4614f39-e56a-47a0-921b-47d781557007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:37:42.387188 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4614f39-e56a-47a0-921b-47d781557007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:37:42.387464 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4614f39-e56a-47a0-921b-47d781557007 admin admin] PATH_INFO: `/v3/users/1a76a5d15bc1428483ffe66837f914be` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:37:42.405918 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4614f39-e56a-47a0-921b-47d781557007 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a76a5d15bc1428483ffe66837f914be)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:37:42.408061 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4614f39-e56a-47a0-921b-47d781557007 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:37:42.641814 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a4614f39-e56a-47a0-921b-47d781557007 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a76a5d15bc1428483ffe66837f914be'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:42.728668 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a4614f39-e56a-47a0-921b-47d781557007 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a76a5d15bc1428483ffe66837f914be'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:42.729267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a4614f39-e56a-47a0-921b-47d781557007 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a76a5d15bc1428483ffe66837f914be'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:42.729858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a4614f39-e56a-47a0-921b-47d781557007 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a76a5d15bc1428483ffe66837f914be'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:42.737334 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a4614f39-e56a-47a0-921b-47d781557007 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a76a5d15bc1428483ffe66837f914be'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:42.755202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1140/2284] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:37:42 2022] DELETE /identity/v3/users/1a76a5d15bc1428483ffe66837f914be => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:37:43.035767 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597504a-a158-4406-a6fb-cb7f27493278 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:37:43.035767 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7597504a-a158-4406-a6fb-cb7f27493278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:37:43.048692 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597504a-a158-4406-a6fb-cb7f27493278 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:37:43.110703 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd9810-1436-4e2b-9ff0-a6da98a86ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:37:43.111626 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd9810-1436-4e2b-9ff0-a6da98a86ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:37:43.111955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd9810-1436-4e2b-9ff0-a6da98a86ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:37:43.112239 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd9810-1436-4e2b-9ff0-a6da98a86ea4 admin admin] PATH_INFO: `/v3/projects/edaadd118ae94e1589641d0f8fc40745` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:37:43.123006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd9810-1436-4e2b-9ff0-a6da98a86ea4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edaadd118ae94e1589641d0f8fc40745)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:37:43.124933 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd9810-1436-4e2b-9ff0-a6da98a86ea4 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:37:43.236669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:37:43.244191 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5bfd9810-1436-4e2b-9ff0-a6da98a86ea4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edaadd118ae94e1589641d0f8fc40745', 'request_id': 'req-5bfd9810-1436-4e2b-9ff0-a6da98a86ea4'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:43.249531 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5bfd9810-1436-4e2b-9ff0-a6da98a86ea4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edaadd118ae94e1589641d0f8fc40745', 'request_id': 'req-5bfd9810-1436-4e2b-9ff0-a6da98a86ea4'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:43.257103 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5bfd9810-1436-4e2b-9ff0-a6da98a86ea4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edaadd118ae94e1589641d0f8fc40745', 'request_id': 'req-5bfd9810-1436-4e2b-9ff0-a6da98a86ea4'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:43.351440 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5bfd9810-1436-4e2b-9ff0-a6da98a86ea4 admin admin] The token cache is being invalidate because project edaadd118ae94e1589641d0f8fc40745 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:37:43.351813 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5bfd9810-1436-4e2b-9ff0-a6da98a86ea4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:43.352446 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5bfd9810-1436-4e2b-9ff0-a6da98a86ea4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:37:43.354053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1145/2285] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:37:43 2022] DELETE /identity/v3/projects/edaadd118ae94e1589641d0f8fc40745 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:37:52.750982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=84,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:37:53.416579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=10,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:37:55.945674 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4614f39-e56a-47a0-921b-47d781557007 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:37:55.947136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a4614f39-e56a-47a0-921b-47d781557007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:37:56.004292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4614f39-e56a-47a0-921b-47d781557007 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:37:56.054236 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0fda9-4324-4fa5-9ce1-8ba01285c51c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:37:56.054879 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0fda9-4324-4fa5-9ce1-8ba01285c51c service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:37:56.055183 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0fda9-4324-4fa5-9ce1-8ba01285c51c service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:37:56.055478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0fda9-4324-4fa5-9ce1-8ba01285c51c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:37:56.057358 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e7f0fda9-4324-4fa5-9ce1-8ba01285c51c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:37:56.125954 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f0fda9-4324-4fa5-9ce1-8ba01285c51c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:37:56.127429 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f0fda9-4324-4fa5-9ce1-8ba01285c51c service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:37:56.136801 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1141/2286] 10.222.0.27 () {60 vars in 1396 bytes} [Sat Jul 16 15:37:55 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:37:56.509352 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd9810-1436-4e2b-9ff0-a6da98a86ea4 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:37:56.510763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5bfd9810-1436-4e2b-9ff0-a6da98a86ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:37:56.582864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd9810-1436-4e2b-9ff0-a6da98a86ea4 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:37:56.588384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d62dbd-861c-4468-8842-71b934f5eaf9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:37:56.589021 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d62dbd-861c-4468-8842-71b934f5eaf9 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:37:56.589301 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d62dbd-861c-4468-8842-71b934f5eaf9 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:37:56.589572 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d62dbd-861c-4468-8842-71b934f5eaf9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:37:56.591325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-76d62dbd-861c-4468-8842-71b934f5eaf9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:37:56.663864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d62dbd-861c-4468-8842-71b934f5eaf9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:37:56.665500 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d62dbd-861c-4468-8842-71b934f5eaf9 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:37:56.674905 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1146/2287] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:37:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:38:06.132656 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=73 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:38:06.670445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=61 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:39:01.951386 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0fda9-4324-4fa5-9ce1-8ba01285c51c service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:39:01.952773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e7f0fda9-4324-4fa5-9ce1-8ba01285c51c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:39:02.065411 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0fda9-4324-4fa5-9ce1-8ba01285c51c service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:39:02.070848 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771df07-d4a2-4d73-90ca-fcb7c3a466e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a30cee766814c8685578819261705ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:39:02.071522 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771df07-d4a2-4d73-90ca-fcb7c3a466e2 service placement] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:39:02.071809 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771df07-d4a2-4d73-90ca-fcb7c3a466e2 service placement] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:39:02.072086 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771df07-d4a2-4d73-90ca-fcb7c3a466e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:39:02.073854 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a771df07-d4a2-4d73-90ca-fcb7c3a466e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:39:02.085709 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771df07-d4a2-4d73-90ca-fcb7c3a466e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:39:02.087336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771df07-d4a2-4d73-90ca-fcb7c3a466e2 service placement] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:39:02.097352 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1142/2288] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:39:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:39:12.093230 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=52 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:39:32.152864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d62dbd-861c-4468-8842-71b934f5eaf9 service nova] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:39:32.216306 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d62dbd-861c-4468-8842-71b934f5eaf9 service nova] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:39:32.222818 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e96671-4840-45cb-8136-5487fcca1365 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:39:32.223587 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e96671-4840-45cb-8136-5487fcca1365 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:39:32.223872 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e96671-4840-45cb-8136-5487fcca1365 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:39:32.224155 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e96671-4840-45cb-8136-5487fcca1365 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:39:32.226224 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-59e96671-4840-45cb-8136-5487fcca1365 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:39:32.241279 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e96671-4840-45cb-8136-5487fcca1365 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:39:32.243002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e96671-4840-45cb-8136-5487fcca1365 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:39:32.256349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1147/2289] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:39:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:39:42.250154 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=24 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:40:59.610866 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771df07-d4a2-4d73-90ca-fcb7c3a466e2 service placement] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:40:59.612302 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a771df07-d4a2-4d73-90ca-fcb7c3a466e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:40:59.683220 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771df07-d4a2-4d73-90ca-fcb7c3a466e2 service placement] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:40:59.687175 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5825831-3289-4f5b-91b6-71da76415ba0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:40:59.687821 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5825831-3289-4f5b-91b6-71da76415ba0 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:40:59.688090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5825831-3289-4f5b-91b6-71da76415ba0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:40:59.688971 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5825831-3289-4f5b-91b6-71da76415ba0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:40:59.690777 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a5825831-3289-4f5b-91b6-71da76415ba0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:40:59.763195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5825831-3289-4f5b-91b6-71da76415ba0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:40:59.764636 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5825831-3289-4f5b-91b6-71da76415ba0 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:40:59.773622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1143/2290] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:40:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:40:59.821975 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e96671-4840-45cb-8136-5487fcca1365 service nova] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:40:59.830975 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e96671-4840-45cb-8136-5487fcca1365 service nova] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:40:59.834784 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce280bed-08ca-4f2c-b585-2ba0eb508c48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:40:59.835440 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce280bed-08ca-4f2c-b585-2ba0eb508c48 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:40:59.835717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce280bed-08ca-4f2c-b585-2ba0eb508c48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:40:59.835988 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce280bed-08ca-4f2c-b585-2ba0eb508c48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:40:59.837745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ce280bed-08ca-4f2c-b585-2ba0eb508c48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:40:59.910291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce280bed-08ca-4f2c-b585-2ba0eb508c48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:40:59.911851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce280bed-08ca-4f2c-b585-2ba0eb508c48 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:40:59.921879 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1148/2291] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:40:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:41:09.769503 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=57 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:41:09.917206 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=33 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:41:58.161789 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5825831-3289-4f5b-91b6-71da76415ba0 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:41:58.163279 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a5825831-3289-4f5b-91b6-71da76415ba0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:41:58.184549 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5825831-3289-4f5b-91b6-71da76415ba0 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:41:58.188050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f68c0-152e-4244-a91f-e62665857371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:41:58.188684 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f68c0-152e-4244-a91f-e62665857371 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:41:58.188963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f68c0-152e-4244-a91f-e62665857371 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:41:58.189237 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f68c0-152e-4244-a91f-e62665857371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:41:58.190938 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b47f68c0-152e-4244-a91f-e62665857371 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:41:58.216905 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f68c0-152e-4244-a91f-e62665857371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:41:58.218325 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f68c0-152e-4244-a91f-e62665857371 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:41:58.229055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1144/2292] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:41:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:41:59.356210 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce280bed-08ca-4f2c-b585-2ba0eb508c48 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:41:59.369480 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce280bed-08ca-4f2c-b585-2ba0eb508c48 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:41:59.374919 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2b559-46d2-425c-bdba-4eb2c7f49cda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:41:59.375564 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2b559-46d2-425c-bdba-4eb2c7f49cda service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:41:59.375850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2b559-46d2-425c-bdba-4eb2c7f49cda service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:41:59.376128 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2b559-46d2-425c-bdba-4eb2c7f49cda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:41:59.377892 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-f6e2b559-46d2-425c-bdba-4eb2c7f49cda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:41:59.435924 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2b559-46d2-425c-bdba-4eb2c7f49cda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:41:59.437552 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2b559-46d2-425c-bdba-4eb2c7f49cda service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:41:59.447070 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1149/2293] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:41:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:42:08.224296 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=15 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:42:09.442555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=28 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:42:56.164519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f68c0-152e-4244-a91f-e62665857371 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:42:56.173394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f68c0-152e-4244-a91f-e62665857371 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:42:56.176996 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943652c-cdb6-41a3-ae7d-70f8b862c862 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:42:56.177634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943652c-cdb6-41a3-ae7d-70f8b862c862 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:42:56.177918 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943652c-cdb6-41a3-ae7d-70f8b862c862 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:42:56.178344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943652c-cdb6-41a3-ae7d-70f8b862c862 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:42:56.186605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a943652c-cdb6-41a3-ae7d-70f8b862c862 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:42:56.188060 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a943652c-cdb6-41a3-ae7d-70f8b862c862 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:42:56.197126 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1145/2294] 10.222.0.27 () {60 vars in 1396 bytes} [Sat Jul 16 15:42:56 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:42:56.824250 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2b559-46d2-425c-bdba-4eb2c7f49cda service nova] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:42:56.836790 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2b559-46d2-425c-bdba-4eb2c7f49cda service nova] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:42:56.842216 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3a6d9-8b43-4e8f-b707-6988f92945ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:42:56.842854 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3a6d9-8b43-4e8f-b707-6988f92945ab service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:42:56.843163 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3a6d9-8b43-4e8f-b707-6988f92945ab service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:42:56.843440 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3a6d9-8b43-4e8f-b707-6988f92945ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:42:56.852339 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3a6d9-8b43-4e8f-b707-6988f92945ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:42:56.853973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3a6d9-8b43-4e8f-b707-6988f92945ab service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:42:56.862873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1150/2295] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:42:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:43:00.333537 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943652c-cdb6-41a3-ae7d-70f8b862c862 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:43:00.342069 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943652c-cdb6-41a3-ae7d-70f8b862c862 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:43:00.345798 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec07fe-a060-445a-b08e-4202a762ae57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:43:00.346437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec07fe-a060-445a-b08e-4202a762ae57 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:43:00.346744 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec07fe-a060-445a-b08e-4202a762ae57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:43:00.347104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec07fe-a060-445a-b08e-4202a762ae57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:43:00.348907 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7bec07fe-a060-445a-b08e-4202a762ae57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:43:00.362843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application [None req-7bec07fe-a060-445a-b08e-4202a762ae57 service cinder] Could not find project: d771120895ed4423a2dbf25c1c9b2732.: keystone.exception.ProjectNotFound: Could not find project: d771120895ed4423a2dbf25c1c9b2732. Jul 16 15:43:00.362843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 16 15:43:00.362843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 16 15:43:00.362843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 16 15:43:00.362843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 16 15:43:00.362843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 16 15:43:00.362843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 16 15:43:00.362843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 16 15:43:00.362843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 16 15:43:00.362843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 16 15:43:00.362843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 16 15:43:00.362843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 16 15:43:00.362843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 16 15:43:00.362843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 16 15:43:00.362843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 16 15:43:00.364273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 16 15:43:00.364273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 16 15:43:00.364273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 16 15:43:00.364273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 15:43:00.364273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 15:43:00.364273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 16 15:43:00.364273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 16 15:43:00.364273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 15:43:00.364273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 15:43:00.364273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 15:43:00.364273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 15:43:00.364273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 15:43:00.364273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 15:43:00.364273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 15:43:00.364273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 15:43:00.364273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 15:43:00.364273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 15:43:00.365482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 15:43:00.365482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 15:43:00.365482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 15:43:00.365482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 15:43:00.365482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 16 15:43:00.365482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 16 15:43:00.365482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 16 15:43:00.365482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 16 15:43:00.365482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 16 15:43:00.365482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 16 15:43:00.365482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 16 15:43:00.365482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 16 15:43:00.365482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 16 15:43:00.365482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 16 15:43:00.365482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 16 15:43:00.365482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 16 15:43:00.365482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 16 15:43:00.367683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.get_or_create( Jul 16 15:43:00.367683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 16 15:43:00.367683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application with Lock( Jul 16 15:43:00.367683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 16 15:43:00.367683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._enter() Jul 16 15:43:00.367683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 16 15:43:00.367683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 16 15:43:00.367683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 16 15:43:00.367683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.creator() Jul 16 15:43:00.367683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 16 15:43:00.367683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application created_value = creator( Jul 16 15:43:00.367683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 16 15:43:00.367683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 16 15:43:00.367683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 16 15:43:00.367683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 16 15:43:00.367683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 16 15:43:00.367683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 16 15:43:00.368823 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: d771120895ed4423a2dbf25c1c9b2732. Jul 16 15:43:00.368823 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: ERROR keystone.server.flask.application Jul 16 15:43:00.368823 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1146/2296] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:43:00 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 33 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jul 16 15:43:06.858737 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=6 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:43:10.353584 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=10 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:43:15.674704 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3a6d9-8b43-4e8f-b707-6988f92945ab service nova] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:43:15.684368 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3a6d9-8b43-4e8f-b707-6988f92945ab service nova] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:43:15.687903 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c880ecd-abb4-49ac-a6af-bdc01493aae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:43:15.688540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c880ecd-abb4-49ac-a6af-bdc01493aae8 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:43:15.688820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c880ecd-abb4-49ac-a6af-bdc01493aae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:43:15.689096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c880ecd-abb4-49ac-a6af-bdc01493aae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:43:15.690857 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-8c880ecd-abb4-49ac-a6af-bdc01493aae8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:43:15.700449 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c880ecd-abb4-49ac-a6af-bdc01493aae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:43:15.701861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c880ecd-abb4-49ac-a6af-bdc01493aae8 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:43:15.710162 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1151/2297] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:43:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:43:25.706157 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=6 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:43:32.612429 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec07fe-a060-445a-b08e-4202a762ae57 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:43:32.630297 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec07fe-a060-445a-b08e-4202a762ae57 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:43:32.636271 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207d1d8-1e1e-4911-95b0-1a4e9c2202ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:43:32.637089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207d1d8-1e1e-4911-95b0-1a4e9c2202ee service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:43:32.637378 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207d1d8-1e1e-4911-95b0-1a4e9c2202ee service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:43:32.647416 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207d1d8-1e1e-4911-95b0-1a4e9c2202ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:43:32.647416 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a207d1d8-1e1e-4911-95b0-1a4e9c2202ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:43:32.654662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207d1d8-1e1e-4911-95b0-1a4e9c2202ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:43:32.656323 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207d1d8-1e1e-4911-95b0-1a4e9c2202ee service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:43:32.669223 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1147/2298] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:43:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:43:35.210848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe6c07-7ca3-4dad-a680-dbe16cd857f0 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:43:35.211417 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe6c07-7ca3-4dad-a680-dbe16cd857f0 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:43:35.211520 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe6c07-7ca3-4dad-a680-dbe16cd857f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:43:35.252555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-0ffe6c07-7ca3-4dad-a680-dbe16cd857f0 None None] MFA Rules not processed for user `9ee0106af0894acb911831433ca22caf`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:43:35.254169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-0ffe6c07-7ca3-4dad-a680-dbe16cd857f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:43:35.270746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1152/2299] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:43:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:43:35.641739 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207d1d8-1e1e-4911-95b0-1a4e9c2202ee service nova] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:43:35.654462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207d1d8-1e1e-4911-95b0-1a4e9c2202ee service nova] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:43:35.659762 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4363b-b3e9-46ae-95c5-d742d9f898cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:43:35.660547 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4363b-b3e9-46ae-95c5-d742d9f898cb service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:43:35.661041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4363b-b3e9-46ae-95c5-d742d9f898cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:43:35.661520 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4363b-b3e9-46ae-95c5-d742d9f898cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:43:35.663617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-50d4363b-b3e9-46ae-95c5-d742d9f898cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:43:35.674633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4363b-b3e9-46ae-95c5-d742d9f898cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:43:35.676135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4363b-b3e9-46ae-95c5-d742d9f898cb service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:43:35.685112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1148/2300] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:43:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:43:36.162412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe6c07-7ca3-4dad-a680-dbe16cd857f0 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:43:36.163909 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-0ffe6c07-7ca3-4dad-a680-dbe16cd857f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:43:36.180050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe6c07-7ca3-4dad-a680-dbe16cd857f0 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:43:36.185301 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f159d3-0c19-4cb3-9360-039fd3cd1481 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a30cee766814c8685578819261705ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:43:36.185953 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f159d3-0c19-4cb3-9360-039fd3cd1481 service placement] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:43:36.186236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f159d3-0c19-4cb3-9360-039fd3cd1481 service placement] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:43:36.186514 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f159d3-0c19-4cb3-9360-039fd3cd1481 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:43:36.188468 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-47f159d3-0c19-4cb3-9360-039fd3cd1481 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:43:36.201941 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f159d3-0c19-4cb3-9360-039fd3cd1481 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:43:36.206067 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f159d3-0c19-4cb3-9360-039fd3cd1481 service placement] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:43:36.214611 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1153/2301] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:43:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:43:45.642856 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4363b-b3e9-46ae-95c5-d742d9f898cb service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:43:45.655932 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4363b-b3e9-46ae-95c5-d742d9f898cb service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:43:45.661310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333e603-2fd0-49d9-ba14-6b0b31d8b43b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:43:45.661945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333e603-2fd0-49d9-ba14-6b0b31d8b43b service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:43:45.662224 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333e603-2fd0-49d9-ba14-6b0b31d8b43b service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:43:45.662494 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333e603-2fd0-49d9-ba14-6b0b31d8b43b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:43:45.664331 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f333e603-2fd0-49d9-ba14-6b0b31d8b43b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:43:45.677425 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f333e603-2fd0-49d9-ba14-6b0b31d8b43b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:43:45.679090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f333e603-2fd0-49d9-ba14-6b0b31d8b43b service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:43:45.690399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1149/2302] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:43:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:43:46.209695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=20 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:43:55.686209 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=18 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:44:04.977135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f159d3-0c19-4cb3-9360-039fd3cd1481 service placement] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:44:04.988266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f159d3-0c19-4cb3-9360-039fd3cd1481 service placement] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:44:04.993627 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a7200-6099-4edc-b6f7-393501c97f00 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a30cee766814c8685578819261705ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:44:04.994334 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a7200-6099-4edc-b6f7-393501c97f00 service placement] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:44:04.994638 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a7200-6099-4edc-b6f7-393501c97f00 service placement] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:44:04.994945 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a7200-6099-4edc-b6f7-393501c97f00 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:44:05.003725 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a7200-6099-4edc-b6f7-393501c97f00 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:44:05.005303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a7200-6099-4edc-b6f7-393501c97f00 service placement] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:44:05.015963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1154/2303] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:44:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:44:15.011006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=6 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:45:03.568863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333e603-2fd0-49d9-ba14-6b0b31d8b43b service nova] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:45:03.585683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333e603-2fd0-49d9-ba14-6b0b31d8b43b service nova] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:45:03.592016 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f6fd0-8fb9-4b36-84a4-d5580d465433 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:45:03.592797 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f6fd0-8fb9-4b36-84a4-d5580d465433 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:45:03.592991 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f6fd0-8fb9-4b36-84a4-d5580d465433 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:45:03.593262 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f6fd0-8fb9-4b36-84a4-d5580d465433 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:45:03.605064 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f6fd0-8fb9-4b36-84a4-d5580d465433 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:45:03.607721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f6fd0-8fb9-4b36-84a4-d5580d465433 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:45:03.619396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1150/2304] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:45:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:45:13.614164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=6 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:46:00.341609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a7200-6099-4edc-b6f7-393501c97f00 service placement] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:46:00.353550 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a7200-6099-4edc-b6f7-393501c97f00 service placement] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:46:00.358931 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da162d45-3f93-4ce3-99d6-ac4cab06f8c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:46:00.359590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da162d45-3f93-4ce3-99d6-ac4cab06f8c0 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:46:00.359873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da162d45-3f93-4ce3-99d6-ac4cab06f8c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:46:00.360147 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da162d45-3f93-4ce3-99d6-ac4cab06f8c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:46:00.367942 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da162d45-3f93-4ce3-99d6-ac4cab06f8c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:46:00.369510 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da162d45-3f93-4ce3-99d6-ac4cab06f8c0 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:46:00.397251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1155/2305] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:46:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:46:10.394611 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=13 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:46:26.151787 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174b8a8-22c8-45ce-9575-2951f0efab12 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:46:26.152311 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174b8a8-22c8-45ce-9575-2951f0efab12 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:46:26.152469 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174b8a8-22c8-45ce-9575-2951f0efab12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:46:26.180427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-4174b8a8-22c8-45ce-9575-2951f0efab12 None None] MFA Rules not processed for user `d9cf1686980749fabd8f54073573ae86`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:46:26.181643 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4174b8a8-22c8-45ce-9575-2951f0efab12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:46:26.183327 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da162d45-3f93-4ce3-99d6-ac4cab06f8c0 service nova] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:46:26.185570 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-da162d45-3f93-4ce3-99d6-ac4cab06f8c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:46:26.190042 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1151/2306] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:46:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:46:26.197978 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da162d45-3f93-4ce3-99d6-ac4cab06f8c0 service nova] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:46:26.202847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88130c62-f179-48eb-9032-c06b8eadf71d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:46:26.203542 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88130c62-f179-48eb-9032-c06b8eadf71d service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:46:26.203827 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88130c62-f179-48eb-9032-c06b8eadf71d service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:46:26.204110 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88130c62-f179-48eb-9032-c06b8eadf71d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:46:26.213177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88130c62-f179-48eb-9032-c06b8eadf71d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:46:26.214682 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88130c62-f179-48eb-9032-c06b8eadf71d service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:46:26.245645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1156/2307] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:46:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:46:32.358571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bffdc5-d9ed-4a8a-bc29-c1a8e55bffac None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:46:32.359122 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bffdc5-d9ed-4a8a-bc29-c1a8e55bffac None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:46:32.359245 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bffdc5-d9ed-4a8a-bc29-c1a8e55bffac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:46:32.388449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-72bffdc5-d9ed-4a8a-bc29-c1a8e55bffac None None] MFA Rules not processed for user `4cadd4bc4d36437484f495d0e49d9881`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:46:32.389576 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-72bffdc5-d9ed-4a8a-bc29-c1a8e55bffac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:46:32.396520 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1152/2308] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:46:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:46:36.242474 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=13 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:46:42.385504 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=12 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:46:46.740260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd9a86-963d-489a-be54-804dcd5e4b98 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:46:46.741710 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd9a86-963d-489a-be54-804dcd5e4b98 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:46:46.742190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd9a86-963d-489a-be54-804dcd5e4b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:46:46.827904 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-42cd9a86-963d-489a-be54-804dcd5e4b98 None None] MFA Rules not processed for user `a4f499707998403d8f833eb783599466`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:46:46.829403 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-42cd9a86-963d-489a-be54-804dcd5e4b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:46:46.912551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1157/2309] 10.222.0.27 () {58 vars in 1009 bytes} [Sat Jul 16 15:46:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:46:47.023866 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bffdc5-d9ed-4a8a-bc29-c1a8e55bffac None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:46:47.026025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-72bffdc5-d9ed-4a8a-bc29-c1a8e55bffac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:46:47.135719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bffdc5-d9ed-4a8a-bc29-c1a8e55bffac None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:46:47.147055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e6c05-1a59-4594-840a-2a7e7a306779 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1464c83fb294a3d90e5763ef648e834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:46:47.148039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e6c05-1a59-4594-840a-2a7e7a306779 service swift] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:46:47.148469 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e6c05-1a59-4594-840a-2a7e7a306779 service swift] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:46:47.149278 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e6c05-1a59-4594-840a-2a7e7a306779 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:46:47.162351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e6c05-1a59-4594-840a-2a7e7a306779 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:46:47.164101 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e6c05-1a59-4594-840a-2a7e7a306779 service swift] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:46:47.175899 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1153/2310] 10.222.0.27 () {60 vars in 1384 bytes} [Sat Jul 16 15:46:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 15:46:56.909250 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=44 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:46:57.170617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=41 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:46:57.498665 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd9a86-963d-489a-be54-804dcd5e4b98 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:46:57.507834 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd9a86-963d-489a-be54-804dcd5e4b98 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:46:57.511653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e2734-f10d-4e89-ac92-32c9e7209717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:46:57.515522 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e2734-f10d-4e89-ac92-32c9e7209717 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:46:57.515522 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e2734-f10d-4e89-ac92-32c9e7209717 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:46:57.515522 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e2734-f10d-4e89-ac92-32c9e7209717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:46:57.523495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e2734-f10d-4e89-ac92-32c9e7209717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:46:57.524951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e2734-f10d-4e89-ac92-32c9e7209717 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:46:57.536957 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1158/2311] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:46:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:46:58.148021 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e6c05-1a59-4594-840a-2a7e7a306779 service swift] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:46:58.157921 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e6c05-1a59-4594-840a-2a7e7a306779 service swift] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:46:58.161960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30ee6c-84c6-40a0-b178-c0105a9fbba9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:46:58.162604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30ee6c-84c6-40a0-b178-c0105a9fbba9 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:46:58.162972 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30ee6c-84c6-40a0-b178-c0105a9fbba9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:46:58.163192 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30ee6c-84c6-40a0-b178-c0105a9fbba9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:46:58.171620 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d30ee6c-84c6-40a0-b178-c0105a9fbba9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:46:58.173046 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d30ee6c-84c6-40a0-b178-c0105a9fbba9 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:46:58.181850 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1154/2312] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:46:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:47:03.489455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6fa785-7194-470c-a308-cb1cafbf9b12 None None] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:03.490092 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6fa785-7194-470c-a308-cb1cafbf9b12 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:03.490092 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6fa785-7194-470c-a308-cb1cafbf9b12 None None] PATH_INFO: `/` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:03.491762 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1159/2313] 10.222.0.27 () {54 vars in 921 bytes} [Sat Jul 16 15:47:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:47:03.498993 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee156c44-b1fe-466a-9166-716b789a4c73 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:03.499439 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee156c44-b1fe-466a-9166-716b789a4c73 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:03.499731 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee156c44-b1fe-466a-9166-716b789a4c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:03.557121 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ee156c44-b1fe-466a-9166-716b789a4c73 None None] MFA Rules not processed for user `3f2e241494244edf8d283e8f44192c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:47:03.558477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ee156c44-b1fe-466a-9166-716b789a4c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:03.568170 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1155/2314] 10.222.0.27 () {58 vars in 1018 bytes} [Sat Jul 16 15:47:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:47:03.580530 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6fa785-7194-470c-a308-cb1cafbf9b12 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:03.582273 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-dc6fa785-7194-470c-a308-cb1cafbf9b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:03.667212 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6fa785-7194-470c-a308-cb1cafbf9b12 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:03.671809 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df61e94-ca00-48c7-964e-21f89f3f503d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d689162291f4f4b97e886ea20d3a8b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:03.672470 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df61e94-ca00-48c7-964e-21f89f3f503d service glance] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:03.672752 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df61e94-ca00-48c7-964e-21f89f3f503d service glance] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:03.673021 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df61e94-ca00-48c7-964e-21f89f3f503d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:03.684062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df61e94-ca00-48c7-964e-21f89f3f503d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:03.685684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df61e94-ca00-48c7-964e-21f89f3f503d service glance] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:03.697548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1160/2315] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:47:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:47:03.893681 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0c2b7-792b-427e-9eec-1fb7bf730777 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:03.893973 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0c2b7-792b-427e-9eec-1fb7bf730777 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:03.894251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0c2b7-792b-427e-9eec-1fb7bf730777 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:03.895680 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1156/2316] 10.222.0.27 () {54 vars in 921 bytes} [Sat Jul 16 15:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:47:03.900617 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847b461-8af7-43f9-b149-ccad157d416f None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:03.900905 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847b461-8af7-43f9-b149-ccad157d416f None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:03.901179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847b461-8af7-43f9-b149-ccad157d416f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:03.939056 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-b847b461-8af7-43f9-b149-ccad157d416f None None] MFA Rules not processed for user `6e372cc62af24664b01a967435eb8902`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:47:03.940310 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b847b461-8af7-43f9-b149-ccad157d416f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:03.947411 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1161/2317] 10.222.0.27 () {58 vars in 1018 bytes} [Sat Jul 16 15:47:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:47:03.959942 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d0c2b7-792b-427e-9eec-1fb7bf730777 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:03.961716 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b3d0c2b7-792b-427e-9eec-1fb7bf730777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:03.975994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d0c2b7-792b-427e-9eec-1fb7bf730777 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:03.979758 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886fc8fe-8e8a-404f-ae35-25a133e3a192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:03.980405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fc8fe-8e8a-404f-ae35-25a133e3a192 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:03.980688 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fc8fe-8e8a-404f-ae35-25a133e3a192 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:03.980966 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fc8fe-8e8a-404f-ae35-25a133e3a192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:03.988659 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886fc8fe-8e8a-404f-ae35-25a133e3a192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:03.990000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886fc8fe-8e8a-404f-ae35-25a133e3a192 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:04.004533 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1157/2318] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:47:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:47:04.065683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847b461-8af7-43f9-b149-ccad157d416f None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:04.074382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847b461-8af7-43f9-b149-ccad157d416f None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:04.078861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ef0c2-c65f-4ede-858b-a24ee7d8eddb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:04.079547 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ef0c2-c65f-4ede-858b-a24ee7d8eddb service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:04.079832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ef0c2-c65f-4ede-858b-a24ee7d8eddb service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:04.080111 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ef0c2-c65f-4ede-858b-a24ee7d8eddb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:04.081944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c22ef0c2-c65f-4ede-858b-a24ee7d8eddb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:04.099827 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611e429-aeca-4fdb-b7be-976c4b3578a3 None None] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:04.100199 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611e429-aeca-4fdb-b7be-976c4b3578a3 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:04.100634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611e429-aeca-4fdb-b7be-976c4b3578a3 None None] PATH_INFO: `/` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:04.102144 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1158/2319] 10.222.0.27 () {54 vars in 921 bytes} [Sat Jul 16 15:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 16 15:47:04.113975 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9f857-fdc9-4481-97a6-28fc43d06a18 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:04.113975 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9f857-fdc9-4481-97a6-28fc43d06a18 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:04.113975 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9f857-fdc9-4481-97a6-28fc43d06a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:04.127122 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c3f9f857-fdc9-4481-97a6-28fc43d06a18 None None] MFA Rules not processed for user `3f2e241494244edf8d283e8f44192c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:47:04.128325 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c3f9f857-fdc9-4481-97a6-28fc43d06a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:04.143536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1159/2320] 10.222.0.27 () {58 vars in 1018 bytes} [Sat Jul 16 15:47:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:47:04.155317 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9f857-fdc9-4481-97a6-28fc43d06a18 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:04.155486 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ef0c2-c65f-4ede-858b-a24ee7d8eddb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:04.156871 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ef0c2-c65f-4ede-858b-a24ee7d8eddb service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:04.165335 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9f857-fdc9-4481-97a6-28fc43d06a18 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:04.166978 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1162/2321] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:47:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:47:04.169904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7bf3b-4085-43be-8445-784b5dd10346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:04.170625 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7bf3b-4085-43be-8445-784b5dd10346 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:04.170941 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7bf3b-4085-43be-8445-784b5dd10346 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:04.171222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7bf3b-4085-43be-8445-784b5dd10346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:04.181906 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb7bf3b-4085-43be-8445-784b5dd10346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:04.183316 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb7bf3b-4085-43be-8445-784b5dd10346 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:04.194333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1160/2322] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:47:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:47:06.300436 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f4320-ab2f-4e3a-b7f6-0a140a958da6 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:06.300995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f4320-ab2f-4e3a-b7f6-0a140a958da6 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:06.300995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f4320-ab2f-4e3a-b7f6-0a140a958da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:06.323292 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-100f4320-ab2f-4e3a-b7f6-0a140a958da6 None None] MFA Rules not processed for user `a4f499707998403d8f833eb783599466`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:47:06.325599 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-100f4320-ab2f-4e3a-b7f6-0a140a958da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:06.339570 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1163/2323] 10.222.0.27 () {58 vars in 1009 bytes} [Sat Jul 16 15:47:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:47:06.349754 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7bf3b-4085-43be-8445-784b5dd10346 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:06.366713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7bf3b-4085-43be-8445-784b5dd10346 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:06.373897 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe98e4-9049-4154-a387-0debf07545f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1464c83fb294a3d90e5763ef648e834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:06.375134 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe98e4-9049-4154-a387-0debf07545f8 service swift] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:06.375613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe98e4-9049-4154-a387-0debf07545f8 service swift] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:06.378518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe98e4-9049-4154-a387-0debf07545f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:06.389669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe98e4-9049-4154-a387-0debf07545f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:06.391514 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe98e4-9049-4154-a387-0debf07545f8 service swift] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:06.401814 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1161/2324] 10.222.0.27 () {60 vars in 1384 bytes} [Sat Jul 16 15:47:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 15:47:09.086199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f4320-ab2f-4e3a-b7f6-0a140a958da6 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:09.106311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f4320-ab2f-4e3a-b7f6-0a140a958da6 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:09.112465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073f2df-2fdd-423a-8d0f-7ba777377e52 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a30cee766814c8685578819261705ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:09.113156 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073f2df-2fdd-423a-8d0f-7ba777377e52 service placement] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:09.117647 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073f2df-2fdd-423a-8d0f-7ba777377e52 service placement] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:09.117647 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073f2df-2fdd-423a-8d0f-7ba777377e52 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:09.117647 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-4073f2df-2fdd-423a-8d0f-7ba777377e52 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:09.130234 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073f2df-2fdd-423a-8d0f-7ba777377e52 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:09.131991 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073f2df-2fdd-423a-8d0f-7ba777377e52 service placement] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:09.152556 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe98e4-9049-4154-a387-0debf07545f8 service swift] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:09.154870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-99fe98e4-9049-4154-a387-0debf07545f8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:09.156259 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1164/2325] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:47:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:47:09.179155 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe98e4-9049-4154-a387-0debf07545f8 service swift] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:09.194621 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4935a21-4e83-4659-a86c-ba7b5ba661bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:09.194621 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4935a21-4e83-4659-a86c-ba7b5ba661bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:09.194621 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4935a21-4e83-4659-a86c-ba7b5ba661bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:09.194621 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4935a21-4e83-4659-a86c-ba7b5ba661bc admin admin] PATH_INFO: `/v3/users/096fd1b2e2b4482da1026d3fb7e192c2` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:09.203782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4935a21-4e83-4659-a86c-ba7b5ba661bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=096fd1b2e2b4482da1026d3fb7e192c2)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:09.205942 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4935a21-4e83-4659-a86c-ba7b5ba661bc admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:09.411056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c4935a21-4e83-4659-a86c-ba7b5ba661bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '096fd1b2e2b4482da1026d3fb7e192c2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:09.474399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c4935a21-4e83-4659-a86c-ba7b5ba661bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '096fd1b2e2b4482da1026d3fb7e192c2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:09.475057 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c4935a21-4e83-4659-a86c-ba7b5ba661bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '096fd1b2e2b4482da1026d3fb7e192c2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:09.475829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c4935a21-4e83-4659-a86c-ba7b5ba661bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '096fd1b2e2b4482da1026d3fb7e192c2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:09.486230 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c4935a21-4e83-4659-a86c-ba7b5ba661bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '096fd1b2e2b4482da1026d3fb7e192c2'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:09.508275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1162/2326] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:47:09 2022] DELETE /identity/v3/users/096fd1b2e2b4482da1026d3fb7e192c2 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:47:09.815130 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073f2df-2fdd-423a-8d0f-7ba777377e52 service placement] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:09.827690 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073f2df-2fdd-423a-8d0f-7ba777377e52 service placement] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:09.850171 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:47:09.888391 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8a26bd-6164-497e-870b-f3d184f7d358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:09.889269 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a26bd-6164-497e-870b-f3d184f7d358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:09.889551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a26bd-6164-497e-870b-f3d184f7d358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:09.889850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a26bd-6164-497e-870b-f3d184f7d358 admin admin] PATH_INFO: `/v3/projects/009b3494f3794d3e8011b85dfe2d2d9b` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:09.891062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8a26bd-6164-497e-870b-f3d184f7d358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=009b3494f3794d3e8011b85dfe2d2d9b)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:09.892984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8a26bd-6164-497e-870b-f3d184f7d358 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:09.997740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-dc8a26bd-6164-497e-870b-f3d184f7d358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '009b3494f3794d3e8011b85dfe2d2d9b', 'request_id': 'req-dc8a26bd-6164-497e-870b-f3d184f7d358'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:10.018299 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-dc8a26bd-6164-497e-870b-f3d184f7d358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '009b3494f3794d3e8011b85dfe2d2d9b', 'request_id': 'req-dc8a26bd-6164-497e-870b-f3d184f7d358'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:10.026910 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-dc8a26bd-6164-497e-870b-f3d184f7d358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '009b3494f3794d3e8011b85dfe2d2d9b', 'request_id': 'req-dc8a26bd-6164-497e-870b-f3d184f7d358'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:10.080029 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-dc8a26bd-6164-497e-870b-f3d184f7d358 admin admin] The token cache is being invalidate because project 009b3494f3794d3e8011b85dfe2d2d9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:47:10.080323 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-dc8a26bd-6164-497e-870b-f3d184f7d358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:10.080950 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-dc8a26bd-6164-497e-870b-f3d184f7d358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:10.082552 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1165/2327] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:47:09 2022] DELETE /identity/v3/projects/009b3494f3794d3e8011b85dfe2d2d9b => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:47:10.088462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286f4a7-2d18-4b7c-8508-5c4d6a65ebe2 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:10.088756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286f4a7-2d18-4b7c-8508-5c4d6a65ebe2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:10.089041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286f4a7-2d18-4b7c-8508-5c4d6a65ebe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:10.152185 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-6286f4a7-2d18-4b7c-8508-5c4d6a65ebe2 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:47:10.154534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6286f4a7-2d18-4b7c-8508-5c4d6a65ebe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:10.174666 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:47:10.217718 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1163/2328] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:47:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:47:10.225454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506fdc9-c887-401f-a8ca-ef862f6d84e7 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:10.225954 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506fdc9-c887-401f-a8ca-ef862f6d84e7 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:10.226082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506fdc9-c887-401f-a8ca-ef862f6d84e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:10.245520 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-3506fdc9-c887-401f-a8ca-ef862f6d84e7 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:47:10.246756 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3506fdc9-c887-401f-a8ca-ef862f6d84e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:10.255041 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1166/2329] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:47:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:47:10.259338 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286f4a7-2d18-4b7c-8508-5c4d6a65ebe2 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:10.269169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286f4a7-2d18-4b7c-8508-5c4d6a65ebe2 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:10.273754 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88351f-e5f3-4c92-bd77-d12b3a2b4b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:10.278817 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88351f-e5f3-4c92-bd77-d12b3a2b4b0e admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:10.278817 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88351f-e5f3-4c92-bd77-d12b3a2b4b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:10.278817 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a88351f-e5f3-4c92-bd77-d12b3a2b4b0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:10.278817 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88351f-e5f3-4c92-bd77-d12b3a2b4b0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:47:10.278817 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88351f-e5f3-4c92-bd77-d12b3a2b4b0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:10.278817 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a88351f-e5f3-4c92-bd77-d12b3a2b4b0e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:10.290321 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1164/2330] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:47:10.295425 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506fdc9-c887-401f-a8ca-ef862f6d84e7 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:10.304855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506fdc9-c887-401f-a8ca-ef862f6d84e7 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:10.309554 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349265d-eee9-4a91-96e0-d34f2e76cd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:10.310199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349265d-eee9-4a91-96e0-d34f2e76cd3e admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:10.310485 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349265d-eee9-4a91-96e0-d34f2e76cd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:10.310762 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349265d-eee9-4a91-96e0-d34f2e76cd3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:10.311498 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349265d-eee9-4a91-96e0-d34f2e76cd3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:10.313407 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349265d-eee9-4a91-96e0-d34f2e76cd3e admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:10.409527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1167/2331] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:47:10 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:47:10.414026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88351f-e5f3-4c92-bd77-d12b3a2b4b0e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:10.425140 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a88351f-e5f3-4c92-bd77-d12b3a2b4b0e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:10.509908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94916c1-0883-4e12-bee5-fbc5b600c54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:10.510817 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94916c1-0883-4e12-bee5-fbc5b600c54b admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:10.511290 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94916c1-0883-4e12-bee5-fbc5b600c54b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:10.511741 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94916c1-0883-4e12-bee5-fbc5b600c54b admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:10.513228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94916c1-0883-4e12-bee5-fbc5b600c54b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:10.515666 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94916c1-0883-4e12-bee5-fbc5b600c54b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:10.603783 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1165/2332] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:47:10 2022] POST /identity/v3/users => generated 287 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:47:10.607960 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349265d-eee9-4a91-96e0-d34f2e76cd3e admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:10.618922 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349265d-eee9-4a91-96e0-d34f2e76cd3e admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:10.623787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c889c34-194f-4f64-ba38-13ff77057be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:10.624444 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c889c34-194f-4f64-ba38-13ff77057be9 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:10.624723 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c889c34-194f-4f64-ba38-13ff77057be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:10.624992 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c889c34-194f-4f64-ba38-13ff77057be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:10.625718 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c889c34-194f-4f64-ba38-13ff77057be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:10.627394 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c889c34-194f-4f64-ba38-13ff77057be9 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:10.640045 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1168/2333] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:47:10.643899 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94916c1-0883-4e12-bee5-fbc5b600c54b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:10.656295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94916c1-0883-4e12-bee5-fbc5b600c54b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:10.661494 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af5af9-6146-452f-8a23-0dd780c57a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:10.662134 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af5af9-6146-452f-8a23-0dd780c57a9b admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:10.662412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af5af9-6146-452f-8a23-0dd780c57a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:10.662678 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af5af9-6146-452f-8a23-0dd780c57a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:10.663449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af5af9-6146-452f-8a23-0dd780c57a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:10.665016 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af5af9-6146-452f-8a23-0dd780c57a9b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:10.678665 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1166/2334] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:47:10.680189 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c889c34-194f-4f64-ba38-13ff77057be9 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:10.691363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c889c34-194f-4f64-ba38-13ff77057be9 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:10.704311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffc4b7-6ab6-4a22-8079-3041efab6cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:10.705938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffc4b7-6ab6-4a22-8079-3041efab6cf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:10.706443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffc4b7-6ab6-4a22-8079-3041efab6cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:10.706722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffc4b7-6ab6-4a22-8079-3041efab6cf9 admin admin] PATH_INFO: `/v3/projects/dd4f213ad117481eb860381684258b4c/users/99f3d2ba65354feb995ebc3748986302/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:10.752897 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ffc4b7-6ab6-4a22-8079-3041efab6cf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd4f213ad117481eb860381684258b4c, user_id=99f3d2ba65354feb995ebc3748986302, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:10.756098 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ffc4b7-6ab6-4a22-8079-3041efab6cf9 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:10.831469 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1169/2335] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:47:10 2022] PUT /identity/v3/projects/dd4f213ad117481eb860381684258b4c/users/99f3d2ba65354feb995ebc3748986302/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:47:10.841549 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870d24f-cc5a-40c0-a9f6-76aff7f55efc None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:10.841844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870d24f-cc5a-40c0-a9f6-76aff7f55efc None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:10.842123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870d24f-cc5a-40c0-a9f6-76aff7f55efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:10.868915 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-9870d24f-cc5a-40c0-a9f6-76aff7f55efc None None] MFA Rules not processed for user `99f3d2ba65354feb995ebc3748986302`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:47:10.870167 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9870d24f-cc5a-40c0-a9f6-76aff7f55efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:10.936860 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1167/2336] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:47:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:47:10.947435 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffc4b7-6ab6-4a22-8079-3041efab6cf9 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:10.949099 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-43ffc4b7-6ab6-4a22-8079-3041efab6cf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:10.994967 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:47:11.021929 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffc4b7-6ab6-4a22-8079-3041efab6cf9 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:11.028627 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ae935-14d7-442a-b9f2-3fe2e5518f83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:11.029266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ae935-14d7-442a-b9f2-3fe2e5518f83 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:11.033063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ae935-14d7-442a-b9f2-3fe2e5518f83 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:11.033063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ae935-14d7-442a-b9f2-3fe2e5518f83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:11.037871 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ae935-14d7-442a-b9f2-3fe2e5518f83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:11.039492 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ae935-14d7-442a-b9f2-3fe2e5518f83 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:11.049002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1170/2337] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:47:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:47:11.280552 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870d24f-cc5a-40c0-a9f6-76aff7f55efc None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:11.299864 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:47:11.343343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870d24f-cc5a-40c0-a9f6-76aff7f55efc None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:11.347833 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df1917-cb28-49a9-8c80-1021f8d1d6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:11.348698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df1917-cb28-49a9-8c80-1021f8d1d6ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:11.348979 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df1917-cb28-49a9-8c80-1021f8d1d6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:11.349253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df1917-cb28-49a9-8c80-1021f8d1d6ca admin admin] PATH_INFO: `/v3/users/99f3d2ba65354feb995ebc3748986302` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:11.350435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df1917-cb28-49a9-8c80-1021f8d1d6ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=99f3d2ba65354feb995ebc3748986302)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:11.352445 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df1917-cb28-49a9-8c80-1021f8d1d6ca admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:11.539075 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c0df1917-cb28-49a9-8c80-1021f8d1d6ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99f3d2ba65354feb995ebc3748986302'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:11.627303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c0df1917-cb28-49a9-8c80-1021f8d1d6ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99f3d2ba65354feb995ebc3748986302'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:11.629530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c0df1917-cb28-49a9-8c80-1021f8d1d6ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99f3d2ba65354feb995ebc3748986302'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:11.630253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c0df1917-cb28-49a9-8c80-1021f8d1d6ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99f3d2ba65354feb995ebc3748986302'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:11.642010 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-c0df1917-cb28-49a9-8c80-1021f8d1d6ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99f3d2ba65354feb995ebc3748986302'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:11.673851 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1168/2338] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:47:11 2022] DELETE /identity/v3/users/99f3d2ba65354feb995ebc3748986302 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:47:11.677892 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110f9f7-011e-48cb-840d-b5ba933ae905 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:11.678636 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110f9f7-011e-48cb-840d-b5ba933ae905 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:11.678636 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110f9f7-011e-48cb-840d-b5ba933ae905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:11.696874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-6110f9f7-011e-48cb-840d-b5ba933ae905 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:47:11.698182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-6110f9f7-011e-48cb-840d-b5ba933ae905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:11.767013 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1171/2339] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:47:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:47:11.777016 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df1917-cb28-49a9-8c80-1021f8d1d6ca admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:11.779488 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c0df1917-cb28-49a9-8c80-1021f8d1d6ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:11.858223 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df1917-cb28-49a9-8c80-1021f8d1d6ca admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:11.862402 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262447b-a99a-4778-a3b8-ec579ea840e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:11.863747 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262447b-a99a-4778-a3b8-ec579ea840e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:11.863747 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262447b-a99a-4778-a3b8-ec579ea840e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:11.863747 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262447b-a99a-4778-a3b8-ec579ea840e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:11.873873 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262447b-a99a-4778-a3b8-ec579ea840e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:11.878509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262447b-a99a-4778-a3b8-ec579ea840e0 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:11.887234 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1169/2340] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:47:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:47:12.391485 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110f9f7-011e-48cb-840d-b5ba933ae905 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:12.392970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-6110f9f7-011e-48cb-840d-b5ba933ae905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:12.407438 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110f9f7-011e-48cb-840d-b5ba933ae905 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:12.413153 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddad4df-f1d2-47ed-b800-7431cf737bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:12.414039 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddad4df-f1d2-47ed-b800-7431cf737bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:12.414473 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddad4df-f1d2-47ed-b800-7431cf737bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:12.414647 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddad4df-f1d2-47ed-b800-7431cf737bfc admin admin] PATH_INFO: `/v3/projects/dd4f213ad117481eb860381684258b4c` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:12.416419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddad4df-f1d2-47ed-b800-7431cf737bfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd4f213ad117481eb860381684258b4c)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:12.417832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddad4df-f1d2-47ed-b800-7431cf737bfc admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:12.522775 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-8ddad4df-f1d2-47ed-b800-7431cf737bfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd4f213ad117481eb860381684258b4c', 'request_id': 'req-8ddad4df-f1d2-47ed-b800-7431cf737bfc'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:12.528553 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-8ddad4df-f1d2-47ed-b800-7431cf737bfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd4f213ad117481eb860381684258b4c', 'request_id': 'req-8ddad4df-f1d2-47ed-b800-7431cf737bfc'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:12.535066 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-8ddad4df-f1d2-47ed-b800-7431cf737bfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd4f213ad117481eb860381684258b4c', 'request_id': 'req-8ddad4df-f1d2-47ed-b800-7431cf737bfc'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:12.607856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-8ddad4df-f1d2-47ed-b800-7431cf737bfc admin admin] The token cache is being invalidate because project dd4f213ad117481eb860381684258b4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:47:12.608535 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-8ddad4df-f1d2-47ed-b800-7431cf737bfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:12.609719 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-8ddad4df-f1d2-47ed-b800-7431cf737bfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:12.611853 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1172/2341] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:47:12 2022] DELETE /identity/v3/projects/dd4f213ad117481eb860381684258b4c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:47:12.616624 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99861483-ea5f-4b61-b774-c5554f4fab79 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:12.617104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99861483-ea5f-4b61-b774-c5554f4fab79 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:12.617194 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99861483-ea5f-4b61-b774-c5554f4fab79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:12.637609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-99861483-ea5f-4b61-b774-c5554f4fab79 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:47:12.638735 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-99861483-ea5f-4b61-b774-c5554f4fab79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:12.665664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:47:12.704184 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1170/2342] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:47:12.713435 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4510e0-a497-4083-beef-8995adeaa702 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:12.713952 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4510e0-a497-4083-beef-8995adeaa702 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:12.714059 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4510e0-a497-4083-beef-8995adeaa702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:12.733623 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-2c4510e0-a497-4083-beef-8995adeaa702 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:47:12.734933 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-2c4510e0-a497-4083-beef-8995adeaa702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:12.744421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1173/2343] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:47:12.748503 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99861483-ea5f-4b61-b774-c5554f4fab79 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:12.763171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99861483-ea5f-4b61-b774-c5554f4fab79 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:12.774535 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6a07f-f688-4240-9a47-a2c673809001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:12.781875 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6a07f-f688-4240-9a47-a2c673809001 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:12.782499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6a07f-f688-4240-9a47-a2c673809001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:12.783044 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6a07f-f688-4240-9a47-a2c673809001 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:12.784390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6a07f-f688-4240-9a47-a2c673809001 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:47:12.784941 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6a07f-f688-4240-9a47-a2c673809001 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:12.787002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6a07f-f688-4240-9a47-a2c673809001 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:12.803635 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1171/2344] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:47:12.808106 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4510e0-a497-4083-beef-8995adeaa702 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:12.819763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4510e0-a497-4083-beef-8995adeaa702 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:12.826349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d0869-588d-4806-8bb3-fb630e530553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:12.827451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d0869-588d-4806-8bb3-fb630e530553 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:12.827979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d0869-588d-4806-8bb3-fb630e530553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:12.828424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d0869-588d-4806-8bb3-fb630e530553 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:12.829454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d0869-588d-4806-8bb3-fb630e530553 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:12.832487 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d0869-588d-4806-8bb3-fb630e530553 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:12.923665 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1174/2345] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:47:12 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:47:12.931107 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6a07f-f688-4240-9a47-a2c673809001 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:12.942017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6a07f-f688-4240-9a47-a2c673809001 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:13.012266 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6b8e9-0b13-49b9-a6ba-723a54613b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:13.013248 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6b8e9-0b13-49b9-a6ba-723a54613b38 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:13.013537 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6b8e9-0b13-49b9-a6ba-723a54613b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:13.013817 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6b8e9-0b13-49b9-a6ba-723a54613b38 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:13.014523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6b8e9-0b13-49b9-a6ba-723a54613b38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:13.016553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6b8e9-0b13-49b9-a6ba-723a54613b38 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:13.098696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1172/2346] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:47:12 2022] POST /identity/v3/users => generated 288 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:47:13.103100 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d0869-588d-4806-8bb3-fb630e530553 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:13.114929 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d0869-588d-4806-8bb3-fb630e530553 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:13.120008 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac68cda-69fa-4136-9cae-f169b34ddaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:13.120879 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac68cda-69fa-4136-9cae-f169b34ddaa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:13.121308 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac68cda-69fa-4136-9cae-f169b34ddaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:13.121674 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac68cda-69fa-4136-9cae-f169b34ddaa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:13.122524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac68cda-69fa-4136-9cae-f169b34ddaa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:13.124295 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac68cda-69fa-4136-9cae-f169b34ddaa0 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:13.135368 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1175/2347] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:47:13.139518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6b8e9-0b13-49b9-a6ba-723a54613b38 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:13.149710 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6b8e9-0b13-49b9-a6ba-723a54613b38 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:13.161110 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66f0e5-bd92-47e1-9c62-0be61b396809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:13.161802 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66f0e5-bd92-47e1-9c62-0be61b396809 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:13.162139 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66f0e5-bd92-47e1-9c62-0be61b396809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:13.162368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66f0e5-bd92-47e1-9c62-0be61b396809 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:13.163162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66f0e5-bd92-47e1-9c62-0be61b396809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:13.167891 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66f0e5-bd92-47e1-9c62-0be61b396809 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:13.190554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1173/2348] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:47:13.192157 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac68cda-69fa-4136-9cae-f169b34ddaa0 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:13.204418 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac68cda-69fa-4136-9cae-f169b34ddaa0 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:13.209903 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f9147-9ed6-445d-8800-300f5d9d976b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:13.211429 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f9147-9ed6-445d-8800-300f5d9d976b admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:13.211429 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f9147-9ed6-445d-8800-300f5d9d976b admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:13.215454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f9147-9ed6-445d-8800-300f5d9d976b admin admin] PATH_INFO: `/v3/projects/8fbee46746634ca79b34dbcae48d50da/users/deeff5290d7140beb683d4b813e9ec5a/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:13.232004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:47:13.243267 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f9147-9ed6-445d-8800-300f5d9d976b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fbee46746634ca79b34dbcae48d50da, user_id=deeff5290d7140beb683d4b813e9ec5a, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:13.246873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f9147-9ed6-445d-8800-300f5d9d976b admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:13.333384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1176/2349] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:47:13 2022] PUT /identity/v3/projects/8fbee46746634ca79b34dbcae48d50da/users/deeff5290d7140beb683d4b813e9ec5a/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:47:13.347766 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f2e57-f373-4fa7-909b-b65097f80dc4 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:13.348058 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f2e57-f373-4fa7-909b-b65097f80dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:13.348343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f2e57-f373-4fa7-909b-b65097f80dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:13.378102 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ca2f2e57-f373-4fa7-909b-b65097f80dc4 None None] MFA Rules not processed for user `deeff5290d7140beb683d4b813e9ec5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:47:13.384769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ca2f2e57-f373-4fa7-909b-b65097f80dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:13.447542 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1174/2350] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:47:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:47:13.453525 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f9147-9ed6-445d-8800-300f5d9d976b admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:13.528643 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f9147-9ed6-445d-8800-300f5d9d976b admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:13.535376 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900f10e-21a2-4535-9fcd-31393f731413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:13.536257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900f10e-21a2-4535-9fcd-31393f731413 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:13.536668 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900f10e-21a2-4535-9fcd-31393f731413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:13.536951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900f10e-21a2-4535-9fcd-31393f731413 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:13.537730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2900f10e-21a2-4535-9fcd-31393f731413 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:13.539904 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2900f10e-21a2-4535-9fcd-31393f731413 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:14.413453 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1177/2351] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:47:13 2022] POST /identity/v3/projects => generated 375 bytes in 959 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:47:14.416461 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f2e57-f373-4fa7-909b-b65097f80dc4 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:14.442495 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f2e57-f373-4fa7-909b-b65097f80dc4 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:14.473658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:47:14.513287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980e950-d89d-4cd9-bb38-689e17ec94d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:14.513964 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980e950-d89d-4cd9-bb38-689e17ec94d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:14.514247 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980e950-d89d-4cd9-bb38-689e17ec94d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:14.514522 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980e950-d89d-4cd9-bb38-689e17ec94d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:14.515250 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980e950-d89d-4cd9-bb38-689e17ec94d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:14.517187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980e950-d89d-4cd9-bb38-689e17ec94d2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:14.617088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1175/2352] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:47:14 2022] POST /identity/v3/users => generated 294 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:47:14.621047 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900f10e-21a2-4535-9fcd-31393f731413 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:14.632974 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900f10e-21a2-4535-9fcd-31393f731413 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:14.638084 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0133e6-3f01-488f-a936-4b3347cbad43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:14.638792 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0133e6-3f01-488f-a936-4b3347cbad43 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:14.639095 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0133e6-3f01-488f-a936-4b3347cbad43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:14.639363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0133e6-3f01-488f-a936-4b3347cbad43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:14.640075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0133e6-3f01-488f-a936-4b3347cbad43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:14.641694 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0133e6-3f01-488f-a936-4b3347cbad43 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:14.652146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1178/2353] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:47:14.656886 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980e950-d89d-4cd9-bb38-689e17ec94d2 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:14.667003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980e950-d89d-4cd9-bb38-689e17ec94d2 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:14.672321 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9d6b2-c9d0-4e30-9668-914e76e1c5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:14.673196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9d6b2-c9d0-4e30-9668-914e76e1c5d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:14.673478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9d6b2-c9d0-4e30-9668-914e76e1c5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:14.673753 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9d6b2-c9d0-4e30-9668-914e76e1c5d1 admin admin] PATH_INFO: `/v3/projects/7d5902c864eb48c9a92c4b5eba8d3335/users/f931f0bd65a04ee684ae29660b9a34e8/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:14.688486 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9d6b2-c9d0-4e30-9668-914e76e1c5d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d5902c864eb48c9a92c4b5eba8d3335, user_id=f931f0bd65a04ee684ae29660b9a34e8, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:14.691154 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9d6b2-c9d0-4e30-9668-914e76e1c5d1 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:14.772003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1176/2354] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:47:14 2022] PUT /identity/v3/projects/7d5902c864eb48c9a92c4b5eba8d3335/users/f931f0bd65a04ee684ae29660b9a34e8/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:47:14.776143 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0133e6-3f01-488f-a936-4b3347cbad43 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:14.787913 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0133e6-3f01-488f-a936-4b3347cbad43 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:14.854664 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1295e8-9371-46ac-802b-0bdc9a074d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:14.855383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1295e8-9371-46ac-802b-0bdc9a074d32 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:14.856422 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1295e8-9371-46ac-802b-0bdc9a074d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:14.856757 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1295e8-9371-46ac-802b-0bdc9a074d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:14.857490 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1295e8-9371-46ac-802b-0bdc9a074d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:14.859115 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1295e8-9371-46ac-802b-0bdc9a074d32 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:14.869657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1179/2355] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:47:14.874837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9d6b2-c9d0-4e30-9668-914e76e1c5d1 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:14.886599 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9d6b2-c9d0-4e30-9668-914e76e1c5d1 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:14.892301 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2d613-e967-47fc-8593-1dee1cb5f442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:14.893316 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2d613-e967-47fc-8593-1dee1cb5f442 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:14.893671 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2d613-e967-47fc-8593-1dee1cb5f442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:14.894017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2d613-e967-47fc-8593-1dee1cb5f442 admin admin] PATH_INFO: `/v3/projects/7d5902c864eb48c9a92c4b5eba8d3335/users/f931f0bd65a04ee684ae29660b9a34e8/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:14.896331 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2d613-e967-47fc-8593-1dee1cb5f442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d5902c864eb48c9a92c4b5eba8d3335, user_id=f931f0bd65a04ee684ae29660b9a34e8, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:14.903083 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2d613-e967-47fc-8593-1dee1cb5f442 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:14.912667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1177/2356] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:47:14 2022] PUT /identity/v3/projects/7d5902c864eb48c9a92c4b5eba8d3335/users/f931f0bd65a04ee684ae29660b9a34e8/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:47:14.926817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde1b9f-c3df-474d-ba64-28358110d6d0 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:14.926817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde1b9f-c3df-474d-ba64-28358110d6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:14.926817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde1b9f-c3df-474d-ba64-28358110d6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:14.953386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-4dde1b9f-c3df-474d-ba64-28358110d6d0 None None] MFA Rules not processed for user `f931f0bd65a04ee684ae29660b9a34e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:47:14.965879 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-4dde1b9f-c3df-474d-ba64-28358110d6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:15.002246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:47:15.026265 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1180/2357] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:47:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:47:15.039477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2d613-e967-47fc-8593-1dee1cb5f442 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:15.045686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9bc2d613-e967-47fc-8593-1dee1cb5f442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:15.172680 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2d613-e967-47fc-8593-1dee1cb5f442 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:15.176982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97bf22-05a3-4c31-ba83-6917d95b160f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:15.177864 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97bf22-05a3-4c31-ba83-6917d95b160f service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:15.178315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97bf22-05a3-4c31-ba83-6917d95b160f service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:15.178757 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97bf22-05a3-4c31-ba83-6917d95b160f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:15.242244 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97bf22-05a3-4c31-ba83-6917d95b160f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:15.243723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97bf22-05a3-4c31-ba83-6917d95b160f service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:15.253605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1178/2358] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:47:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:47:21.624609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde1b9f-c3df-474d-ba64-28358110d6d0 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:21.626008 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-4dde1b9f-c3df-474d-ba64-28358110d6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:21.638252 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde1b9f-c3df-474d-ba64-28358110d6d0 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:21.641776 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba7caf-782f-4709-a8d3-46db9ddb4b6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:21.642408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba7caf-782f-4709-a8d3-46db9ddb4b6f service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:21.642688 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba7caf-782f-4709-a8d3-46db9ddb4b6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:21.642998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba7caf-782f-4709-a8d3-46db9ddb4b6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:21.651761 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba7caf-782f-4709-a8d3-46db9ddb4b6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:21.653225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba7caf-782f-4709-a8d3-46db9ddb4b6f service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:21.662843 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1181/2359] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:47:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:47:25.249391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=86,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:47:31.658344 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=14 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:47:35.659131 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ec57f-87cb-4a77-9d36-7064a4d90254 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:35.660809 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ec57f-87cb-4a77-9d36-7064a4d90254 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:35.660809 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ec57f-87cb-4a77-9d36-7064a4d90254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:35.689318 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d80ec57f-87cb-4a77-9d36-7064a4d90254 None None] MFA Rules not processed for user `a4f499707998403d8f833eb783599466`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:47:35.690524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d80ec57f-87cb-4a77-9d36-7064a4d90254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:35.771928 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1179/2360] 10.222.0.27 () {58 vars in 967 bytes} [Sat Jul 16 15:47:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:47:35.772997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba7caf-782f-4709-a8d3-46db9ddb4b6f service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:35.774524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-adba7caf-782f-4709-a8d3-46db9ddb4b6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:35.861619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba7caf-782f-4709-a8d3-46db9ddb4b6f service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:35.866846 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d0754-f69e-4c95-b59d-7024e5839f42 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1464c83fb294a3d90e5763ef648e834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:35.867504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d0754-f69e-4c95-b59d-7024e5839f42 service swift] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:35.867786 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d0754-f69e-4c95-b59d-7024e5839f42 service swift] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:35.868061 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d0754-f69e-4c95-b59d-7024e5839f42 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:35.877758 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d0754-f69e-4c95-b59d-7024e5839f42 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:35.878428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d0754-f69e-4c95-b59d-7024e5839f42 service swift] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:35.887443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1182/2361] 10.222.0.27 () {60 vars in 1384 bytes} [Sat Jul 16 15:47:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 15:47:44.322246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ec57f-87cb-4a77-9d36-7064a4d90254 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:44.324184 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d80ec57f-87cb-4a77-9d36-7064a4d90254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:44.379889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ec57f-87cb-4a77-9d36-7064a4d90254 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:44.383599 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e1dd9-4074-4f72-bde6-a53af2ebe639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:44.384239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e1dd9-4074-4f72-bde6-a53af2ebe639 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:44.384521 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e1dd9-4074-4f72-bde6-a53af2ebe639 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:44.384819 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e1dd9-4074-4f72-bde6-a53af2ebe639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:44.386639 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f69e1dd9-4074-4f72-bde6-a53af2ebe639 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:44.468459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e1dd9-4074-4f72-bde6-a53af2ebe639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:44.469883 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e1dd9-4074-4f72-bde6-a53af2ebe639 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:44.489403 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1180/2362] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:47:45.259016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d0754-f69e-4c95-b59d-7024e5839f42 service swift] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:45.261624 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-9a9d0754-f69e-4c95-b59d-7024e5839f42 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:45.341584 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d0754-f69e-4c95-b59d-7024e5839f42 service swift] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:45.346647 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0fd529-e408-42d9-a4e3-2b5a0de3a043 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a30cee766814c8685578819261705ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:45.347322 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0fd529-e408-42d9-a4e3-2b5a0de3a043 service placement] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:45.347603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0fd529-e408-42d9-a4e3-2b5a0de3a043 service placement] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:45.347883 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0fd529-e408-42d9-a4e3-2b5a0de3a043 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:45.349650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5c0fd529-e408-42d9-a4e3-2b5a0de3a043 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:45.361993 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0fd529-e408-42d9-a4e3-2b5a0de3a043 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:45.363605 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0fd529-e408-42d9-a4e3-2b5a0de3a043 service placement] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:45.373585 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1183/2363] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:47:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:47:45.389467 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e1dd9-4074-4f72-bde6-a53af2ebe639 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:45.391131 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f69e1dd9-4074-4f72-bde6-a53af2ebe639 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:45.404058 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e1dd9-4074-4f72-bde6-a53af2ebe639 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:45.408950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3705a0d-2d26-4a72-8915-ade5601fee36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:45.410390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3705a0d-2d26-4a72-8915-ade5601fee36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:45.410668 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3705a0d-2d26-4a72-8915-ade5601fee36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:45.411070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3705a0d-2d26-4a72-8915-ade5601fee36 admin admin] PATH_INFO: `/v3/users/d9cf1686980749fabd8f54073573ae86` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:45.414344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:47:45.428854 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3705a0d-2d26-4a72-8915-ade5601fee36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9cf1686980749fabd8f54073573ae86)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:45.430879 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3705a0d-2d26-4a72-8915-ade5601fee36 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:45.587630 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f3705a0d-2d26-4a72-8915-ade5601fee36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9cf1686980749fabd8f54073573ae86'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:45.648534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f3705a0d-2d26-4a72-8915-ade5601fee36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9cf1686980749fabd8f54073573ae86'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:45.649154 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f3705a0d-2d26-4a72-8915-ade5601fee36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9cf1686980749fabd8f54073573ae86'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:45.649758 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f3705a0d-2d26-4a72-8915-ade5601fee36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9cf1686980749fabd8f54073573ae86'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:45.658186 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f3705a0d-2d26-4a72-8915-ade5601fee36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9cf1686980749fabd8f54073573ae86'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:45.675288 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1181/2364] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:47:45 2022] DELETE /identity/v3/users/d9cf1686980749fabd8f54073573ae86 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:47:45.901739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0fd529-e408-42d9-a4e3-2b5a0de3a043 service placement] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:45.913811 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0fd529-e408-42d9-a4e3-2b5a0de3a043 service placement] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:45.970898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc8bbb-23e3-4fb7-a71d-de811ee41d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:45.971789 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc8bbb-23e3-4fb7-a71d-de811ee41d45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:45.972073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc8bbb-23e3-4fb7-a71d-de811ee41d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:45.972348 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc8bbb-23e3-4fb7-a71d-de811ee41d45 admin admin] PATH_INFO: `/v3/projects/0d5fd17d843c4933b68c201ce7940d5b` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:45.983269 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc8bbb-23e3-4fb7-a71d-de811ee41d45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d5fd17d843c4933b68c201ce7940d5b)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:45.985207 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc8bbb-23e3-4fb7-a71d-de811ee41d45 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:45.991186 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:47:46.083712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-f4cc8bbb-23e3-4fb7-a71d-de811ee41d45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d5fd17d843c4933b68c201ce7940d5b', 'request_id': 'req-f4cc8bbb-23e3-4fb7-a71d-de811ee41d45'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:46.089921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-f4cc8bbb-23e3-4fb7-a71d-de811ee41d45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d5fd17d843c4933b68c201ce7940d5b', 'request_id': 'req-f4cc8bbb-23e3-4fb7-a71d-de811ee41d45'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:46.096662 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-f4cc8bbb-23e3-4fb7-a71d-de811ee41d45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d5fd17d843c4933b68c201ce7940d5b', 'request_id': 'req-f4cc8bbb-23e3-4fb7-a71d-de811ee41d45'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:46.185298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-f4cc8bbb-23e3-4fb7-a71d-de811ee41d45 admin admin] The token cache is being invalidate because project 0d5fd17d843c4933b68c201ce7940d5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:47:46.185519 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-f4cc8bbb-23e3-4fb7-a71d-de811ee41d45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:46.186238 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-f4cc8bbb-23e3-4fb7-a71d-de811ee41d45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:46.187851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1184/2365] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:47:45 2022] DELETE /identity/v3/projects/0d5fd17d843c4933b68c201ce7940d5b => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:47:52.047913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3705a0d-2d26-4a72-8915-ade5601fee36 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:52.049347 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f3705a0d-2d26-4a72-8915-ade5601fee36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:52.062651 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3705a0d-2d26-4a72-8915-ade5601fee36 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:52.116769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773bd1de-e035-4b0b-b078-b48d2839c7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:52.117633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bd1de-e035-4b0b-b078-b48d2839c7fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:52.117915 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bd1de-e035-4b0b-b078-b48d2839c7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:52.118182 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773bd1de-e035-4b0b-b078-b48d2839c7fd admin admin] PATH_INFO: `/v3/users/deeff5290d7140beb683d4b813e9ec5a` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:52.119369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773bd1de-e035-4b0b-b078-b48d2839c7fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=deeff5290d7140beb683d4b813e9ec5a)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:52.121846 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773bd1de-e035-4b0b-b078-b48d2839c7fd admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:52.341118 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-773bd1de-e035-4b0b-b078-b48d2839c7fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deeff5290d7140beb683d4b813e9ec5a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:52.419577 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-773bd1de-e035-4b0b-b078-b48d2839c7fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deeff5290d7140beb683d4b813e9ec5a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:52.420206 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-773bd1de-e035-4b0b-b078-b48d2839c7fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deeff5290d7140beb683d4b813e9ec5a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:52.420868 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-773bd1de-e035-4b0b-b078-b48d2839c7fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deeff5290d7140beb683d4b813e9ec5a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:52.428322 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-773bd1de-e035-4b0b-b078-b48d2839c7fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deeff5290d7140beb683d4b813e9ec5a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:52.435934 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:47:52.446231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1182/2366] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:47:52 2022] DELETE /identity/v3/users/deeff5290d7140beb683d4b813e9ec5a => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:47:52.450738 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e06c7a-8b8f-4e8c-bea9-eedd6540e67e None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:52.451271 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e06c7a-8b8f-4e8c-bea9-eedd6540e67e None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:52.451403 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e06c7a-8b8f-4e8c-bea9-eedd6540e67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:52.468050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-32e06c7a-8b8f-4e8c-bea9-eedd6540e67e None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:47:52.469239 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-32e06c7a-8b8f-4e8c-bea9-eedd6540e67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:52.526411 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1185/2367] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:47:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:47:52.536548 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773bd1de-e035-4b0b-b078-b48d2839c7fd admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:52.537925 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-773bd1de-e035-4b0b-b078-b48d2839c7fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:52.595033 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773bd1de-e035-4b0b-b078-b48d2839c7fd admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:52.598448 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1d96a-a1eb-4153-98c1-6f9f65a0c366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:52.599120 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1d96a-a1eb-4153-98c1-6f9f65a0c366 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:52.599407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1d96a-a1eb-4153-98c1-6f9f65a0c366 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:52.599680 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1d96a-a1eb-4153-98c1-6f9f65a0c366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:52.610011 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1d96a-a1eb-4153-98c1-6f9f65a0c366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:52.611483 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f1d96a-a1eb-4153-98c1-6f9f65a0c366 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:52.621803 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1183/2368] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:47:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:47:53.068731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e06c7a-8b8f-4e8c-bea9-eedd6540e67e None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:53.070237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-32e06c7a-8b8f-4e8c-bea9-eedd6540e67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:53.083977 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e06c7a-8b8f-4e8c-bea9-eedd6540e67e None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:53.089249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad895d-02b5-4473-ad0e-142a15f1e2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:53.090219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad895d-02b5-4473-ad0e-142a15f1e2c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:53.090501 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad895d-02b5-4473-ad0e-142a15f1e2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:53.090779 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad895d-02b5-4473-ad0e-142a15f1e2c9 admin admin] PATH_INFO: `/v3/projects/8fbee46746634ca79b34dbcae48d50da` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:53.092020 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad895d-02b5-4473-ad0e-142a15f1e2c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fbee46746634ca79b34dbcae48d50da)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:53.093995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad895d-02b5-4473-ad0e-142a15f1e2c9 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:53.250504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5dad895d-02b5-4473-ad0e-142a15f1e2c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fbee46746634ca79b34dbcae48d50da', 'request_id': 'req-5dad895d-02b5-4473-ad0e-142a15f1e2c9'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:53.255995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5dad895d-02b5-4473-ad0e-142a15f1e2c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fbee46746634ca79b34dbcae48d50da', 'request_id': 'req-5dad895d-02b5-4473-ad0e-142a15f1e2c9'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:53.262993 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5dad895d-02b5-4473-ad0e-142a15f1e2c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fbee46746634ca79b34dbcae48d50da', 'request_id': 'req-5dad895d-02b5-4473-ad0e-142a15f1e2c9'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:53.357745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5dad895d-02b5-4473-ad0e-142a15f1e2c9 admin admin] The token cache is being invalidate because project 8fbee46746634ca79b34dbcae48d50da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:47:53.358053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5dad895d-02b5-4473-ad0e-142a15f1e2c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:53.358747 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5dad895d-02b5-4473-ad0e-142a15f1e2c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:53.360378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1186/2369] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:47:53 2022] DELETE /identity/v3/projects/8fbee46746634ca79b34dbcae48d50da => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:47:53.364384 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1d96a-a1eb-4153-98c1-6f9f65a0c366 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:53.374132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f1d96a-a1eb-4153-98c1-6f9f65a0c366 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:53.431829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c057c-cc7e-4cec-8005-4b92336ac651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:53.432748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c057c-cc7e-4cec-8005-4b92336ac651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:53.433073 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c057c-cc7e-4cec-8005-4b92336ac651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:53.433399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c057c-cc7e-4cec-8005-4b92336ac651 admin admin] PATH_INFO: `/v3/users/f931f0bd65a04ee684ae29660b9a34e8` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:53.434956 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c057c-cc7e-4cec-8005-4b92336ac651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f931f0bd65a04ee684ae29660b9a34e8)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:53.436873 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c057c-cc7e-4cec-8005-4b92336ac651 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:53.658627 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e94c057c-cc7e-4cec-8005-4b92336ac651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f931f0bd65a04ee684ae29660b9a34e8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:53.735821 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e94c057c-cc7e-4cec-8005-4b92336ac651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f931f0bd65a04ee684ae29660b9a34e8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:53.736437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e94c057c-cc7e-4cec-8005-4b92336ac651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f931f0bd65a04ee684ae29660b9a34e8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:53.737048 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e94c057c-cc7e-4cec-8005-4b92336ac651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f931f0bd65a04ee684ae29660b9a34e8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:53.744331 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-e94c057c-cc7e-4cec-8005-4b92336ac651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f931f0bd65a04ee684ae29660b9a34e8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:53.763606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1184/2370] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:47:53 2022] DELETE /identity/v3/users/f931f0bd65a04ee684ae29660b9a34e8 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:47:54.183018 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad895d-02b5-4473-ad0e-142a15f1e2c9 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:54.184539 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5dad895d-02b5-4473-ad0e-142a15f1e2c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:54.198766 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad895d-02b5-4473-ad0e-142a15f1e2c9 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:54.212399 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:47:54.257423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efca55-6a32-4128-acc5-21405bdcf4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:54.258236 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efca55-6a32-4128-acc5-21405bdcf4a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:54.258523 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efca55-6a32-4128-acc5-21405bdcf4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:54.258800 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efca55-6a32-4128-acc5-21405bdcf4a4 admin admin] PATH_INFO: `/v3/projects/7d5902c864eb48c9a92c4b5eba8d3335` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:54.260032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efca55-6a32-4128-acc5-21405bdcf4a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d5902c864eb48c9a92c4b5eba8d3335)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:54.262133 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efca55-6a32-4128-acc5-21405bdcf4a4 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:54.392150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a1efca55-6a32-4128-acc5-21405bdcf4a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d5902c864eb48c9a92c4b5eba8d3335', 'request_id': 'req-a1efca55-6a32-4128-acc5-21405bdcf4a4'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:54.397551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a1efca55-6a32-4128-acc5-21405bdcf4a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d5902c864eb48c9a92c4b5eba8d3335', 'request_id': 'req-a1efca55-6a32-4128-acc5-21405bdcf4a4'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:54.404172 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a1efca55-6a32-4128-acc5-21405bdcf4a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d5902c864eb48c9a92c4b5eba8d3335', 'request_id': 'req-a1efca55-6a32-4128-acc5-21405bdcf4a4'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:54.484069 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a1efca55-6a32-4128-acc5-21405bdcf4a4 admin admin] The token cache is being invalidate because project 7d5902c864eb48c9a92c4b5eba8d3335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:47:54.484363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a1efca55-6a32-4128-acc5-21405bdcf4a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:54.485019 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a1efca55-6a32-4128-acc5-21405bdcf4a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:47:54.486568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1187/2371] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:47:54 2022] DELETE /identity/v3/projects/7d5902c864eb48c9a92c4b5eba8d3335 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:47:54.491607 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6198d2-ec4a-4e64-81f5-f134eabceb96 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:54.492088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6198d2-ec4a-4e64-81f5-f134eabceb96 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:54.492197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6198d2-ec4a-4e64-81f5-f134eabceb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:54.511601 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3b6198d2-ec4a-4e64-81f5-f134eabceb96 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:47:54.512526 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3b6198d2-ec4a-4e64-81f5-f134eabceb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:54.518397 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:47:54.576334 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1185/2372] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:47:54.584539 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1214dfb-43b7-4b76-9d18-02822459f1ba None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:54.584858 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1214dfb-43b7-4b76-9d18-02822459f1ba None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:54.585163 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1214dfb-43b7-4b76-9d18-02822459f1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:54.605125 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-a1214dfb-43b7-4b76-9d18-02822459f1ba None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:47:54.606336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a1214dfb-43b7-4b76-9d18-02822459f1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:54.614478 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1188/2373] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:47:54.623467 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6198d2-ec4a-4e64-81f5-f134eabceb96 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:54.633525 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6198d2-ec4a-4e64-81f5-f134eabceb96 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:54.639145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99279322-6fac-457b-8e12-47b0dc06be0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:54.639781 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99279322-6fac-457b-8e12-47b0dc06be0b admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:54.640063 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99279322-6fac-457b-8e12-47b0dc06be0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:54.640351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99279322-6fac-457b-8e12-47b0dc06be0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:54.641062 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99279322-6fac-457b-8e12-47b0dc06be0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:47:54.641369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99279322-6fac-457b-8e12-47b0dc06be0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:54.643067 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99279322-6fac-457b-8e12-47b0dc06be0b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:54.659572 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1186/2374] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:47:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:47:54.660772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1214dfb-43b7-4b76-9d18-02822459f1ba None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:54.672296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1214dfb-43b7-4b76-9d18-02822459f1ba None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:54.675487 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b22da-4755-4d58-b980-dfa76cb53160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:54.676145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b22da-4755-4d58-b980-dfa76cb53160 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:54.676472 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b22da-4755-4d58-b980-dfa76cb53160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:54.676734 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b22da-4755-4d58-b980-dfa76cb53160 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:54.677529 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b22da-4755-4d58-b980-dfa76cb53160 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:54.679709 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b22da-4755-4d58-b980-dfa76cb53160 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:54.777428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1189/2375] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:47:54 2022] POST /identity/v3/projects => generated 369 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:47:54.781831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99279322-6fac-457b-8e12-47b0dc06be0b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:54.792503 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99279322-6fac-457b-8e12-47b0dc06be0b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:54.849284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2cd6ac-f41d-4b81-8abd-7e8e4c78e96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:54.849953 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cd6ac-f41d-4b81-8abd-7e8e4c78e96b admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:54.850237 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cd6ac-f41d-4b81-8abd-7e8e4c78e96b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:54.850515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cd6ac-f41d-4b81-8abd-7e8e4c78e96b admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:54.851240 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2cd6ac-f41d-4b81-8abd-7e8e4c78e96b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:54.853186 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2cd6ac-f41d-4b81-8abd-7e8e4c78e96b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:54.952693 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1187/2376] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:47:54 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:47:54.956779 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b22da-4755-4d58-b980-dfa76cb53160 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:54.967769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b22da-4755-4d58-b980-dfa76cb53160 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:54.972449 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306ef1c-6979-4f65-a5a3-dcf402c00ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:54.973071 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306ef1c-6979-4f65-a5a3-dcf402c00ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:54.973357 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306ef1c-6979-4f65-a5a3-dcf402c00ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:54.973635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d306ef1c-6979-4f65-a5a3-dcf402c00ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:54.974360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306ef1c-6979-4f65-a5a3-dcf402c00ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:54.975994 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d306ef1c-6979-4f65-a5a3-dcf402c00ae0 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:54.987272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1190/2377] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:47:54.991459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2cd6ac-f41d-4b81-8abd-7e8e4c78e96b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:55.001685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2cd6ac-f41d-4b81-8abd-7e8e4c78e96b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:55.006360 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8edd4-a1d4-4e59-bb9e-f36f79d458c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:55.007046 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8edd4-a1d4-4e59-bb9e-f36f79d458c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:55.007334 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8edd4-a1d4-4e59-bb9e-f36f79d458c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:55.007612 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8edd4-a1d4-4e59-bb9e-f36f79d458c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:55.008336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c8edd4-a1d4-4e59-bb9e-f36f79d458c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:55.009960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c8edd4-a1d4-4e59-bb9e-f36f79d458c3 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:55.020531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1188/2378] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:47:55.024794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306ef1c-6979-4f65-a5a3-dcf402c00ae0 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:55.036063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d306ef1c-6979-4f65-a5a3-dcf402c00ae0 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:55.040695 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b910229-4e69-4344-953a-8449b83e7238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:55.043758 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b910229-4e69-4344-953a-8449b83e7238 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:55.043758 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b910229-4e69-4344-953a-8449b83e7238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:55.043758 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b910229-4e69-4344-953a-8449b83e7238 admin admin] PATH_INFO: `/v3/projects/defa551f47dc4ea7963d8e1e53e74687/users/f3b355428f364d048cc68865c0f6d739/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:55.056694 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b910229-4e69-4344-953a-8449b83e7238 admin admin] RBAC: Authorizing `identity:create_grant(project_id=defa551f47dc4ea7963d8e1e53e74687, user_id=f3b355428f364d048cc68865c0f6d739, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:55.060051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b910229-4e69-4344-953a-8449b83e7238 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:55.136031 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1191/2379] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:47:55 2022] PUT /identity/v3/projects/defa551f47dc4ea7963d8e1e53e74687/users/f3b355428f364d048cc68865c0f6d739/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:47:55.146484 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d494807-28e7-421e-8062-022f00f60270 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:55.146775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d494807-28e7-421e-8062-022f00f60270 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:55.147150 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d494807-28e7-421e-8062-022f00f60270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:55.175379 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5d494807-28e7-421e-8062-022f00f60270 None None] MFA Rules not processed for user `f3b355428f364d048cc68865c0f6d739`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:47:55.176889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5d494807-28e7-421e-8062-022f00f60270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:55.243583 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1189/2380] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:47:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:47:55.248770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b910229-4e69-4344-953a-8449b83e7238 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:55.298382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:47:55.316403 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b910229-4e69-4344-953a-8449b83e7238 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:55.321175 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f239d9-6df6-4a60-9625-bad051210b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:55.321836 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f239d9-6df6-4a60-9625-bad051210b99 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:55.322120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f239d9-6df6-4a60-9625-bad051210b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:55.322399 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f239d9-6df6-4a60-9625-bad051210b99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:55.323793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f239d9-6df6-4a60-9625-bad051210b99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:55.325714 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f239d9-6df6-4a60-9625-bad051210b99 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:55.427476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1192/2381] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:47:55 2022] POST /identity/v3/projects => generated 367 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:47:55.431602 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d494807-28e7-421e-8062-022f00f60270 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:55.443428 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d494807-28e7-421e-8062-022f00f60270 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:55.447063 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:47:55.500168 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddb838-b072-45df-9166-25283e46ab8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:55.500844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddb838-b072-45df-9166-25283e46ab8f admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:55.501050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddb838-b072-45df-9166-25283e46ab8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:55.501322 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddb838-b072-45df-9166-25283e46ab8f admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:55.502005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ddb838-b072-45df-9166-25283e46ab8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:55.503951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ddb838-b072-45df-9166-25283e46ab8f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:55.632606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1190/2382] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:47:55 2022] POST /identity/v3/users => generated 290 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:47:55.637493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f239d9-6df6-4a60-9625-bad051210b99 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:55.648477 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f239d9-6df6-4a60-9625-bad051210b99 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:55.653086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f74f6e-5590-43b8-9207-03097224235a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:55.653739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f74f6e-5590-43b8-9207-03097224235a admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:55.654022 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f74f6e-5590-43b8-9207-03097224235a admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:55.654299 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f74f6e-5590-43b8-9207-03097224235a admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:55.655057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f74f6e-5590-43b8-9207-03097224235a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:55.656672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f74f6e-5590-43b8-9207-03097224235a admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:55.667796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1193/2383] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:47:55.673688 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddb838-b072-45df-9166-25283e46ab8f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:55.684348 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddb838-b072-45df-9166-25283e46ab8f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:55.689039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba6ace-0917-40c9-8db0-2678d43d709e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:55.689995 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba6ace-0917-40c9-8db0-2678d43d709e admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:55.690299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba6ace-0917-40c9-8db0-2678d43d709e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:55.690586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba6ace-0917-40c9-8db0-2678d43d709e admin admin] PATH_INFO: `/v3/projects/985317702f7b4a7e9154b9c3e3c65266/users/f2b355695b8c477eb11c682297143f40/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:55.705641 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba6ace-0917-40c9-8db0-2678d43d709e admin admin] RBAC: Authorizing `identity:create_grant(project_id=985317702f7b4a7e9154b9c3e3c65266, user_id=f2b355695b8c477eb11c682297143f40, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:55.708335 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba6ace-0917-40c9-8db0-2678d43d709e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:55.787844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1191/2384] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:47:55 2022] PUT /identity/v3/projects/985317702f7b4a7e9154b9c3e3c65266/users/f2b355695b8c477eb11c682297143f40/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:47:55.794325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f74f6e-5590-43b8-9207-03097224235a admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:55.804383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f74f6e-5590-43b8-9207-03097224235a admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:55.860197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1db64-b888-4b77-be5e-2016fae426b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:55.860856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1db64-b888-4b77-be5e-2016fae426b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:55.861140 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1db64-b888-4b77-be5e-2016fae426b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:55.861526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1db64-b888-4b77-be5e-2016fae426b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:55.862252 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1db64-b888-4b77-be5e-2016fae426b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:55.863875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1db64-b888-4b77-be5e-2016fae426b8 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:55.873649 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1194/2385] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:47:55 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:47:55.877872 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba6ace-0917-40c9-8db0-2678d43d709e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:55.888346 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba6ace-0917-40c9-8db0-2678d43d709e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:55.893007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10e079-ba38-4f59-9ce8-0e412b9171b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:55.893891 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10e079-ba38-4f59-9ce8-0e412b9171b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:55.894169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10e079-ba38-4f59-9ce8-0e412b9171b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:55.894445 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10e079-ba38-4f59-9ce8-0e412b9171b1 admin admin] PATH_INFO: `/v3/projects/985317702f7b4a7e9154b9c3e3c65266/users/f2b355695b8c477eb11c682297143f40/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:55.896480 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10e079-ba38-4f59-9ce8-0e412b9171b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=985317702f7b4a7e9154b9c3e3c65266, user_id=f2b355695b8c477eb11c682297143f40, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:55.899153 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10e079-ba38-4f59-9ce8-0e412b9171b1 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:55.911038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1192/2386] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:47:55 2022] PUT /identity/v3/projects/985317702f7b4a7e9154b9c3e3c65266/users/f2b355695b8c477eb11c682297143f40/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:47:55.922772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08272573-418c-43ba-acae-3634c384b0c2 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:55.923166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08272573-418c-43ba-acae-3634c384b0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:55.923381 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08272573-418c-43ba-acae-3634c384b0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:55.949736 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-08272573-418c-43ba-acae-3634c384b0c2 None None] MFA Rules not processed for user `f2b355695b8c477eb11c682297143f40`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:47:55.951039 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-08272573-418c-43ba-acae-3634c384b0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:56.019297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1195/2387] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:47:55 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:47:56.031121 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10e079-ba38-4f59-9ce8-0e412b9171b1 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:56.033048 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ec10e079-ba38-4f59-9ce8-0e412b9171b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:56.090017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10e079-ba38-4f59-9ce8-0e412b9171b1 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:56.093656 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334c966-0c6a-414b-a468-3087f329b505 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:56.094295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334c966-0c6a-414b-a468-3087f329b505 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:56.094580 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334c966-0c6a-414b-a468-3087f329b505 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:56.094860 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334c966-0c6a-414b-a468-3087f329b505 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:56.103825 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334c966-0c6a-414b-a468-3087f329b505 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:56.105889 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334c966-0c6a-414b-a468-3087f329b505 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:56.115272 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1193/2388] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:47:56 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:47:56.235450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08272573-418c-43ba-acae-3634c384b0c2 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:56.245739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08272573-418c-43ba-acae-3634c384b0c2 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:56.250129 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb58b73-a268-43a3-a161-754597e05c0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:56.250129 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb58b73-a268-43a3-a161-754597e05c0d service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:56.251028 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb58b73-a268-43a3-a161-754597e05c0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:56.251028 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb58b73-a268-43a3-a161-754597e05c0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:56.291621 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:47:56.309919 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb58b73-a268-43a3-a161-754597e05c0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:56.311503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb58b73-a268-43a3-a161-754597e05c0d service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:56.320909 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1196/2389] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:47:56 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:47:56.850502 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334c966-0c6a-414b-a468-3087f329b505 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:47:56.852036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6334c966-0c6a-414b-a468-3087f329b505 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:56.863485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334c966-0c6a-414b-a468-3087f329b505 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:47:56.875966 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d8c55-1e75-4c9c-ba40-34e9fbf4ac12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:47:56.875966 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d8c55-1e75-4c9c-ba40-34e9fbf4ac12 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:47:56.875966 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d8c55-1e75-4c9c-ba40-34e9fbf4ac12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:47:56.875966 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d8c55-1e75-4c9c-ba40-34e9fbf4ac12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:47:56.876536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-cc1d8c55-1e75-4c9c-ba40-34e9fbf4ac12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:47:56.951881 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:47:56.971383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d8c55-1e75-4c9c-ba40-34e9fbf4ac12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:47:56.972828 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d8c55-1e75-4c9c-ba40-34e9fbf4ac12 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:47:56.992731 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1194/2390] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:47:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:48:01.080257 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb58b73-a268-43a3-a161-754597e05c0d service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:01.142729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb58b73-a268-43a3-a161-754597e05c0d service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:01.147185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf5e40-ac41-402c-b172-defd5d96900b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:01.148041 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf5e40-ac41-402c-b172-defd5d96900b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:01.148320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf5e40-ac41-402c-b172-defd5d96900b admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:01.148587 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf5e40-ac41-402c-b172-defd5d96900b admin admin] PATH_INFO: `/v3/users/f3b355428f364d048cc68865c0f6d739` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:01.149766 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf5e40-ac41-402c-b172-defd5d96900b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3b355428f364d048cc68865c0f6d739)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:01.152181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf5e40-ac41-402c-b172-defd5d96900b admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:01.387757 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e7bf5e40-ac41-402c-b172-defd5d96900b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3b355428f364d048cc68865c0f6d739'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:01.467166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e7bf5e40-ac41-402c-b172-defd5d96900b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3b355428f364d048cc68865c0f6d739'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:01.474605 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e7bf5e40-ac41-402c-b172-defd5d96900b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3b355428f364d048cc68865c0f6d739'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:01.487108 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e7bf5e40-ac41-402c-b172-defd5d96900b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3b355428f364d048cc68865c0f6d739'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:01.487794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e7bf5e40-ac41-402c-b172-defd5d96900b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3b355428f364d048cc68865c0f6d739'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:01.493752 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1197/2391] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:01 2022] DELETE /identity/v3/users/f3b355428f364d048cc68865c0f6d739 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:01.498571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e920877-05e3-4e9a-8c91-4074453520cb None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:01.499061 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e920877-05e3-4e9a-8c91-4074453520cb None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:01.499181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e920877-05e3-4e9a-8c91-4074453520cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:01.516953 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-4e920877-05e3-4e9a-8c91-4074453520cb None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:01.518141 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4e920877-05e3-4e9a-8c91-4074453520cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:01.575722 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1195/2392] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:01.585439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf5e40-ac41-402c-b172-defd5d96900b admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:01.588868 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e7bf5e40-ac41-402c-b172-defd5d96900b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:01.643926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf5e40-ac41-402c-b172-defd5d96900b admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:01.647516 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479f5ca-d167-4d2b-801a-35cf9e0ff3c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:01.648155 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479f5ca-d167-4d2b-801a-35cf9e0ff3c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:01.648440 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479f5ca-d167-4d2b-801a-35cf9e0ff3c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:01.648718 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479f5ca-d167-4d2b-801a-35cf9e0ff3c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:01.658387 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479f5ca-d167-4d2b-801a-35cf9e0ff3c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:01.659848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479f5ca-d167-4d2b-801a-35cf9e0ff3c4 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:01.664739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:01.669858 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1198/2393] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:48:02.136024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e920877-05e3-4e9a-8c91-4074453520cb None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:02.137613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4e920877-05e3-4e9a-8c91-4074453520cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:02.149968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e920877-05e3-4e9a-8c91-4074453520cb None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:02.155598 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56782d80-fe75-4fd0-bc26-12cf1ccf45cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:02.156344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56782d80-fe75-4fd0-bc26-12cf1ccf45cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:02.156652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56782d80-fe75-4fd0-bc26-12cf1ccf45cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:02.156928 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56782d80-fe75-4fd0-bc26-12cf1ccf45cb admin admin] PATH_INFO: `/v3/projects/defa551f47dc4ea7963d8e1e53e74687` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:02.158375 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56782d80-fe75-4fd0-bc26-12cf1ccf45cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=defa551f47dc4ea7963d8e1e53e74687)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:02.160314 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56782d80-fe75-4fd0-bc26-12cf1ccf45cb admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:02.300348 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-56782d80-fe75-4fd0-bc26-12cf1ccf45cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'defa551f47dc4ea7963d8e1e53e74687', 'request_id': 'req-56782d80-fe75-4fd0-bc26-12cf1ccf45cb'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:02.305904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-56782d80-fe75-4fd0-bc26-12cf1ccf45cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'defa551f47dc4ea7963d8e1e53e74687', 'request_id': 'req-56782d80-fe75-4fd0-bc26-12cf1ccf45cb'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:02.312478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-56782d80-fe75-4fd0-bc26-12cf1ccf45cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'defa551f47dc4ea7963d8e1e53e74687', 'request_id': 'req-56782d80-fe75-4fd0-bc26-12cf1ccf45cb'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:02.392300 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-56782d80-fe75-4fd0-bc26-12cf1ccf45cb admin admin] The token cache is being invalidate because project defa551f47dc4ea7963d8e1e53e74687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:02.392515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-56782d80-fe75-4fd0-bc26-12cf1ccf45cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:02.393135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-56782d80-fe75-4fd0-bc26-12cf1ccf45cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:02.394668 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1196/2394] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:02 2022] DELETE /identity/v3/projects/defa551f47dc4ea7963d8e1e53e74687 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:02.398620 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479f5ca-d167-4d2b-801a-35cf9e0ff3c4 service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:02.409354 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479f5ca-d167-4d2b-801a-35cf9e0ff3c4 service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:02.465135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5e673-0ce3-4b9a-86db-1d82f7851a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:02.465997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5e673-0ce3-4b9a-86db-1d82f7851a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:02.466281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5e673-0ce3-4b9a-86db-1d82f7851a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:02.466557 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5e673-0ce3-4b9a-86db-1d82f7851a83 admin admin] PATH_INFO: `/v3/users/f2b355695b8c477eb11c682297143f40` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:02.467769 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5e673-0ce3-4b9a-86db-1d82f7851a83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2b355695b8c477eb11c682297143f40)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:02.469710 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5e673-0ce3-4b9a-86db-1d82f7851a83 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:02.643787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d3f5e673-0ce3-4b9a-86db-1d82f7851a83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2b355695b8c477eb11c682297143f40'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:02.727482 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d3f5e673-0ce3-4b9a-86db-1d82f7851a83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2b355695b8c477eb11c682297143f40'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:02.734646 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d3f5e673-0ce3-4b9a-86db-1d82f7851a83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2b355695b8c477eb11c682297143f40'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:02.747600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d3f5e673-0ce3-4b9a-86db-1d82f7851a83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2b355695b8c477eb11c682297143f40'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:02.748261 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d3f5e673-0ce3-4b9a-86db-1d82f7851a83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2b355695b8c477eb11c682297143f40'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:02.754203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1199/2395] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:02 2022] DELETE /identity/v3/users/f2b355695b8c477eb11c682297143f40 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:03.153686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56782d80-fe75-4fd0-bc26-12cf1ccf45cb admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:03.155098 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-56782d80-fe75-4fd0-bc26-12cf1ccf45cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:03.169048 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56782d80-fe75-4fd0-bc26-12cf1ccf45cb admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:03.223738 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:03.231108 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00afb69-d1ee-4b0f-a10b-554756789550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:03.231987 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00afb69-d1ee-4b0f-a10b-554756789550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:03.232303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00afb69-d1ee-4b0f-a10b-554756789550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:03.232648 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00afb69-d1ee-4b0f-a10b-554756789550 admin admin] PATH_INFO: `/v3/projects/985317702f7b4a7e9154b9c3e3c65266` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:03.233881 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00afb69-d1ee-4b0f-a10b-554756789550 admin admin] RBAC: Authorizing `identity:delete_project(project_id=985317702f7b4a7e9154b9c3e3c65266)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:03.235892 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00afb69-d1ee-4b0f-a10b-554756789550 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:03.394453 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d00afb69-d1ee-4b0f-a10b-554756789550 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '985317702f7b4a7e9154b9c3e3c65266', 'request_id': 'req-d00afb69-d1ee-4b0f-a10b-554756789550'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:03.400402 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d00afb69-d1ee-4b0f-a10b-554756789550 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '985317702f7b4a7e9154b9c3e3c65266', 'request_id': 'req-d00afb69-d1ee-4b0f-a10b-554756789550'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:03.408068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d00afb69-d1ee-4b0f-a10b-554756789550 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '985317702f7b4a7e9154b9c3e3c65266', 'request_id': 'req-d00afb69-d1ee-4b0f-a10b-554756789550'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:03.477660 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d00afb69-d1ee-4b0f-a10b-554756789550 admin admin] The token cache is being invalidate because project 985317702f7b4a7e9154b9c3e3c65266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:03.478013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d00afb69-d1ee-4b0f-a10b-554756789550 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:03.478691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d00afb69-d1ee-4b0f-a10b-554756789550 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:03.480334 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1197/2396] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:03 2022] DELETE /identity/v3/projects/985317702f7b4a7e9154b9c3e3c65266 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:03.486212 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf094d-b2ee-48e6-afd5-33af34fbc9f8 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:03.486691 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf094d-b2ee-48e6-afd5-33af34fbc9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:03.486795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaf094d-b2ee-48e6-afd5-33af34fbc9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:03.505324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-2aaf094d-b2ee-48e6-afd5-33af34fbc9f8 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:03.506514 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-2aaf094d-b2ee-48e6-afd5-33af34fbc9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:03.565186 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1200/2397] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:03.574392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d89364-7c17-4b32-8205-34275448cb42 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:03.574680 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d89364-7c17-4b32-8205-34275448cb42 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:03.574997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d89364-7c17-4b32-8205-34275448cb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:03.593556 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-98d89364-7c17-4b32-8205-34275448cb42 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:03.594747 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-98d89364-7c17-4b32-8205-34275448cb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:03.602690 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1198/2398] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:03.607457 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf094d-b2ee-48e6-afd5-33af34fbc9f8 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:03.617058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaf094d-b2ee-48e6-afd5-33af34fbc9f8 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:03.622167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4696b39-f105-4280-87b8-8e4f416bf1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:03.622806 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4696b39-f105-4280-87b8-8e4f416bf1fb admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:03.623116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4696b39-f105-4280-87b8-8e4f416bf1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:03.623388 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4696b39-f105-4280-87b8-8e4f416bf1fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:03.624109 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4696b39-f105-4280-87b8-8e4f416bf1fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:48:03.624410 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4696b39-f105-4280-87b8-8e4f416bf1fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:03.625979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4696b39-f105-4280-87b8-8e4f416bf1fb admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:03.637798 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1201/2399] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:48:03.641931 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d89364-7c17-4b32-8205-34275448cb42 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:03.651744 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d89364-7c17-4b32-8205-34275448cb42 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:03.656481 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb580e-6cf8-4815-b992-a04d4b0dc734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:03.657378 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb580e-6cf8-4815-b992-a04d4b0dc734 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:03.657657 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb580e-6cf8-4815-b992-a04d4b0dc734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:03.657924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb580e-6cf8-4815-b992-a04d4b0dc734 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:03.658663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb580e-6cf8-4815-b992-a04d4b0dc734 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:03.660552 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb580e-6cf8-4815-b992-a04d4b0dc734 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:03.753569 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1199/2400] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:03 2022] POST /identity/v3/projects => generated 407 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:03.757562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4696b39-f105-4280-87b8-8e4f416bf1fb admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:03.767706 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4696b39-f105-4280-87b8-8e4f416bf1fb admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:03.771260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:03.824486 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda321b-f00a-4c97-a236-b2272845ef18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:03.825161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda321b-f00a-4c97-a236-b2272845ef18 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:03.825447 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda321b-f00a-4c97-a236-b2272845ef18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:03.825724 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda321b-f00a-4c97-a236-b2272845ef18 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:03.826438 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda321b-f00a-4c97-a236-b2272845ef18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:03.828386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda321b-f00a-4c97-a236-b2272845ef18 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:03.890754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1202/2401] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:03 2022] POST /identity/v3/users => generated 304 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:03.894802 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb580e-6cf8-4815-b992-a04d4b0dc734 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:03.905946 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb580e-6cf8-4815-b992-a04d4b0dc734 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:03.912444 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b3bc5-535a-467a-a247-0a9ad035c3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:03.912444 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b3bc5-535a-467a-a247-0a9ad035c3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:03.912444 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b3bc5-535a-467a-a247-0a9ad035c3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:03.912444 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b3bc5-535a-467a-a247-0a9ad035c3f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:03.913036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b3bc5-535a-467a-a247-0a9ad035c3f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:03.914623 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b3bc5-535a-467a-a247-0a9ad035c3f8 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:03.926427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1200/2402] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:03.930446 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda321b-f00a-4c97-a236-b2272845ef18 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:03.941497 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda321b-f00a-4c97-a236-b2272845ef18 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:03.946196 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684d40c-b20a-4c22-bab5-20dddbdd394e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:03.946854 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684d40c-b20a-4c22-bab5-20dddbdd394e admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:03.947165 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684d40c-b20a-4c22-bab5-20dddbdd394e admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:03.947441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684d40c-b20a-4c22-bab5-20dddbdd394e admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:03.948172 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684d40c-b20a-4c22-bab5-20dddbdd394e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:03.949776 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684d40c-b20a-4c22-bab5-20dddbdd394e admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:03.960868 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1203/2403] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:03.964990 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b3bc5-535a-467a-a247-0a9ad035c3f8 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:03.974983 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b3bc5-535a-467a-a247-0a9ad035c3f8 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:03.980632 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b094ea-5f00-4bb8-b8a9-1cd65d4ac8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:03.981512 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b094ea-5f00-4bb8-b8a9-1cd65d4ac8df admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:03.981790 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b094ea-5f00-4bb8-b8a9-1cd65d4ac8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:03.982062 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b094ea-5f00-4bb8-b8a9-1cd65d4ac8df admin admin] PATH_INFO: `/v3/projects/ee99cfedc53f49a7bad81a670d3ba7a5/users/0454c5eed18c489cbdabe6e011a18e4c/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:03.997797 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b094ea-5f00-4bb8-b8a9-1cd65d4ac8df admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee99cfedc53f49a7bad81a670d3ba7a5, user_id=0454c5eed18c489cbdabe6e011a18e4c, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:04.003304 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b094ea-5f00-4bb8-b8a9-1cd65d4ac8df admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:04.107171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1201/2404] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:03 2022] PUT /identity/v3/projects/ee99cfedc53f49a7bad81a670d3ba7a5/users/0454c5eed18c489cbdabe6e011a18e4c/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:04.117781 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd2c26-f58e-4974-b3a4-aec078b2602e None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:04.118086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd2c26-f58e-4974-b3a4-aec078b2602e None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:04.118352 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd2c26-f58e-4974-b3a4-aec078b2602e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:04.144891 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-b5bd2c26-f58e-4974-b3a4-aec078b2602e None None] MFA Rules not processed for user `0454c5eed18c489cbdabe6e011a18e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:04.146076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b5bd2c26-f58e-4974-b3a4-aec078b2602e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:04.211525 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1204/2405] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:04 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:04.216449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b094ea-5f00-4bb8-b8a9-1cd65d4ac8df admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:04.281669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b094ea-5f00-4bb8-b8a9-1cd65d4ac8df admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:04.286624 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e39e4-86fd-4579-adf1-0baa2bcfb8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:04.287316 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e39e4-86fd-4579-adf1-0baa2bcfb8af admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:04.290139 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e39e4-86fd-4579-adf1-0baa2bcfb8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:04.290139 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e39e4-86fd-4579-adf1-0baa2bcfb8af admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:04.290139 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e39e4-86fd-4579-adf1-0baa2bcfb8af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:04.290554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e39e4-86fd-4579-adf1-0baa2bcfb8af admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:04.375545 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1202/2406] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:04 2022] POST /identity/v3/projects => generated 409 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:04.376545 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd2c26-f58e-4974-b3a4-aec078b2602e None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:04.388171 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd2c26-f58e-4974-b3a4-aec078b2602e None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:04.444575 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f05a5-363c-4b3a-8c35-34b35ccdf7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:04.445248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f05a5-363c-4b3a-8c35-34b35ccdf7a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:04.445531 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f05a5-363c-4b3a-8c35-34b35ccdf7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:04.445806 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2f05a5-363c-4b3a-8c35-34b35ccdf7a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:04.446497 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f05a5-363c-4b3a-8c35-34b35ccdf7a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:04.448702 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2f05a5-363c-4b3a-8c35-34b35ccdf7a2 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:04.521787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1205/2407] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:04 2022] POST /identity/v3/users => generated 311 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:48:04.522816 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e39e4-86fd-4579-adf1-0baa2bcfb8af admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:04.533714 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e39e4-86fd-4579-adf1-0baa2bcfb8af admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:04.538264 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e895b6-5826-490f-9874-cdc02396a292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:04.538945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e895b6-5826-490f-9874-cdc02396a292 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:04.539229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e895b6-5826-490f-9874-cdc02396a292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:04.539702 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e895b6-5826-490f-9874-cdc02396a292 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:04.540479 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e895b6-5826-490f-9874-cdc02396a292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:04.543173 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e895b6-5826-490f-9874-cdc02396a292 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:04.544612 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:04.554517 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1203/2408] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:48:04.559395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f05a5-363c-4b3a-8c35-34b35ccdf7a2 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:04.565928 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:04.569835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2f05a5-363c-4b3a-8c35-34b35ccdf7a2 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:04.574741 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b5f7f-429b-4175-a4b4-a35509b1f554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:04.575751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b5f7f-429b-4175-a4b4-a35509b1f554 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:04.576104 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b5f7f-429b-4175-a4b4-a35509b1f554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:04.576496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b5f7f-429b-4175-a4b4-a35509b1f554 admin admin] PATH_INFO: `/v3/projects/66699060047d4f2f904e871c3c5ff873/users/59b99e1c26964900aef55f28407463a3/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:04.594064 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b5f7f-429b-4175-a4b4-a35509b1f554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66699060047d4f2f904e871c3c5ff873, user_id=59b99e1c26964900aef55f28407463a3, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:04.603069 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b5f7f-429b-4175-a4b4-a35509b1f554 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:04.676944 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1206/2409] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:04 2022] PUT /identity/v3/projects/66699060047d4f2f904e871c3c5ff873/users/59b99e1c26964900aef55f28407463a3/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:04.680964 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e895b6-5826-490f-9874-cdc02396a292 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:04.692061 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e895b6-5826-490f-9874-cdc02396a292 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:04.750679 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4212f3e-e683-4ab8-ae5e-4a1cb277d08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:04.751388 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4212f3e-e683-4ab8-ae5e-4a1cb277d08a admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:04.751674 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4212f3e-e683-4ab8-ae5e-4a1cb277d08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:04.751957 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4212f3e-e683-4ab8-ae5e-4a1cb277d08a admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:04.752688 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4212f3e-e683-4ab8-ae5e-4a1cb277d08a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:04.754283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4212f3e-e683-4ab8-ae5e-4a1cb277d08a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:04.767069 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1204/2410] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:04.769066 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b5f7f-429b-4175-a4b4-a35509b1f554 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:04.781080 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b5f7f-429b-4175-a4b4-a35509b1f554 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:04.786172 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c695fc-0522-4715-bd06-a1cbca97247d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:04.787111 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c695fc-0522-4715-bd06-a1cbca97247d admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:04.787394 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c695fc-0522-4715-bd06-a1cbca97247d admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:04.787700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c695fc-0522-4715-bd06-a1cbca97247d admin admin] PATH_INFO: `/v3/projects/66699060047d4f2f904e871c3c5ff873/users/59b99e1c26964900aef55f28407463a3/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:04.789703 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c695fc-0522-4715-bd06-a1cbca97247d admin admin] RBAC: Authorizing `identity:create_grant(project_id=66699060047d4f2f904e871c3c5ff873, user_id=59b99e1c26964900aef55f28407463a3, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:04.793124 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c695fc-0522-4715-bd06-a1cbca97247d admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:04.814253 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1207/2411] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:04 2022] PUT /identity/v3/projects/66699060047d4f2f904e871c3c5ff873/users/59b99e1c26964900aef55f28407463a3/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:04.815867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229aa7b-ff64-418c-9635-29c037a16f19 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:04.816188 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229aa7b-ff64-418c-9635-29c037a16f19 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:04.816472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229aa7b-ff64-418c-9635-29c037a16f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:04.844036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3229aa7b-ff64-418c-9635-29c037a16f19 None None] MFA Rules not processed for user `59b99e1c26964900aef55f28407463a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:04.845240 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3229aa7b-ff64-418c-9635-29c037a16f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:04.917997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1205/2412] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:04 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:04.928882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c695fc-0522-4715-bd06-a1cbca97247d admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:04.930571 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-39c695fc-0522-4715-bd06-a1cbca97247d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:04.991159 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c695fc-0522-4715-bd06-a1cbca97247d admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:04.995909 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad7d7c-81b8-48fc-9366-5f7f7a2a2702 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:04.996835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad7d7c-81b8-48fc-9366-5f7f7a2a2702 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:04.997118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad7d7c-81b8-48fc-9366-5f7f7a2a2702 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:04.997452 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad7d7c-81b8-48fc-9366-5f7f7a2a2702 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:05.007324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad7d7c-81b8-48fc-9366-5f7f7a2a2702 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:05.008813 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad7d7c-81b8-48fc-9366-5f7f7a2a2702 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:05.018946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1208/2413] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:04 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:48:05.400467 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229aa7b-ff64-418c-9635-29c037a16f19 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:05.401956 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3229aa7b-ff64-418c-9635-29c037a16f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:05.414264 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229aa7b-ff64-418c-9635-29c037a16f19 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:05.418398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63943106-3db5-4d5d-960a-6b57c36516bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:05.419213 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63943106-3db5-4d5d-960a-6b57c36516bf service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:05.419505 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63943106-3db5-4d5d-960a-6b57c36516bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:05.419772 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63943106-3db5-4d5d-960a-6b57c36516bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:05.479415 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63943106-3db5-4d5d-960a-6b57c36516bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:05.482003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63943106-3db5-4d5d-960a-6b57c36516bf service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:05.490867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1206/2414] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:05 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:48:11.182875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad7d7c-81b8-48fc-9366-5f7f7a2a2702 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:11.192204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad7d7c-81b8-48fc-9366-5f7f7a2a2702 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:11.195796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f04fd9-61cb-404d-a4be-95e37ffa40f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:11.196422 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f04fd9-61cb-404d-a4be-95e37ffa40f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:11.196703 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f04fd9-61cb-404d-a4be-95e37ffa40f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:11.196975 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f04fd9-61cb-404d-a4be-95e37ffa40f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:11.198822 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b4f04fd9-61cb-404d-a4be-95e37ffa40f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:11.273637 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f04fd9-61cb-404d-a4be-95e37ffa40f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:11.275371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f04fd9-61cb-404d-a4be-95e37ffa40f8 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:11.285281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1209/2415] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:48:15.487243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=93 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:15.613145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63943106-3db5-4d5d-960a-6b57c36516bf service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:15.714446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63943106-3db5-4d5d-960a-6b57c36516bf service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:15.720839 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4631bec-5927-42d2-bb7d-c9b49491622d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:15.722064 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4631bec-5927-42d2-bb7d-c9b49491622d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:15.722431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4631bec-5927-42d2-bb7d-c9b49491622d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:15.722906 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4631bec-5927-42d2-bb7d-c9b49491622d admin admin] PATH_INFO: `/v3/users/0454c5eed18c489cbdabe6e011a18e4c` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:15.724434 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4631bec-5927-42d2-bb7d-c9b49491622d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0454c5eed18c489cbdabe6e011a18e4c)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:15.726744 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4631bec-5927-42d2-bb7d-c9b49491622d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:16.055118 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b4631bec-5927-42d2-bb7d-c9b49491622d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0454c5eed18c489cbdabe6e011a18e4c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:16.112268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b4631bec-5927-42d2-bb7d-c9b49491622d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0454c5eed18c489cbdabe6e011a18e4c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:16.112834 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b4631bec-5927-42d2-bb7d-c9b49491622d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0454c5eed18c489cbdabe6e011a18e4c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:16.113416 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b4631bec-5927-42d2-bb7d-c9b49491622d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0454c5eed18c489cbdabe6e011a18e4c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:16.124255 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b4631bec-5927-42d2-bb7d-c9b49491622d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0454c5eed18c489cbdabe6e011a18e4c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:16.145688 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1207/2416] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:15 2022] DELETE /identity/v3/users/0454c5eed18c489cbdabe6e011a18e4c => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:16.150430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de29e2-6f86-4383-a7d3-250e575f35bf None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:16.157921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de29e2-6f86-4383-a7d3-250e575f35bf None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:16.157921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de29e2-6f86-4383-a7d3-250e575f35bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:16.178488 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-49de29e2-6f86-4383-a7d3-250e575f35bf None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:16.179777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-49de29e2-6f86-4383-a7d3-250e575f35bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:16.281317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:16.297155 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1210/2417] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:16.307832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4631bec-5927-42d2-bb7d-c9b49491622d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:16.309300 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b4631bec-5927-42d2-bb7d-c9b49491622d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:16.438921 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4631bec-5927-42d2-bb7d-c9b49491622d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:16.445404 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac7897-6b1b-4738-9ceb-813fb741fab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:16.446084 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac7897-6b1b-4738-9ceb-813fb741fab0 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:16.446546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac7897-6b1b-4738-9ceb-813fb741fab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:16.446825 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac7897-6b1b-4738-9ceb-813fb741fab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:16.461001 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac7897-6b1b-4738-9ceb-813fb741fab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:16.467269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac7897-6b1b-4738-9ceb-813fb741fab0 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:16.501405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1208/2418] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:48:17.022383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de29e2-6f86-4383-a7d3-250e575f35bf None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:17.025558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-49de29e2-6f86-4383-a7d3-250e575f35bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:17.037848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de29e2-6f86-4383-a7d3-250e575f35bf None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:17.043634 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cc8ff-0b66-4b8e-a998-834ad82772d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:17.045104 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cc8ff-0b66-4b8e-a998-834ad82772d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:17.045383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cc8ff-0b66-4b8e-a998-834ad82772d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:17.045657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cc8ff-0b66-4b8e-a998-834ad82772d1 admin admin] PATH_INFO: `/v3/projects/ee99cfedc53f49a7bad81a670d3ba7a5` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:17.047003 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cc8ff-0b66-4b8e-a998-834ad82772d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee99cfedc53f49a7bad81a670d3ba7a5)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:17.059029 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cc8ff-0b66-4b8e-a998-834ad82772d1 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:17.184137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-583cc8ff-0b66-4b8e-a998-834ad82772d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee99cfedc53f49a7bad81a670d3ba7a5', 'request_id': 'req-583cc8ff-0b66-4b8e-a998-834ad82772d1'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:17.191209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-583cc8ff-0b66-4b8e-a998-834ad82772d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee99cfedc53f49a7bad81a670d3ba7a5', 'request_id': 'req-583cc8ff-0b66-4b8e-a998-834ad82772d1'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:17.198273 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-583cc8ff-0b66-4b8e-a998-834ad82772d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee99cfedc53f49a7bad81a670d3ba7a5', 'request_id': 'req-583cc8ff-0b66-4b8e-a998-834ad82772d1'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:17.279233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-583cc8ff-0b66-4b8e-a998-834ad82772d1 admin admin] The token cache is being invalidate because project ee99cfedc53f49a7bad81a670d3ba7a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:17.280327 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-583cc8ff-0b66-4b8e-a998-834ad82772d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:17.280921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-583cc8ff-0b66-4b8e-a998-834ad82772d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:17.283108 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1211/2419] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:17 2022] DELETE /identity/v3/projects/ee99cfedc53f49a7bad81a670d3ba7a5 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:17.287369 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac7897-6b1b-4738-9ceb-813fb741fab0 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:17.299577 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac7897-6b1b-4738-9ceb-813fb741fab0 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:17.321971 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:17.374865 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce0623-1c21-497f-a955-bc992c473c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:17.375782 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce0623-1c21-497f-a955-bc992c473c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:17.376065 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce0623-1c21-497f-a955-bc992c473c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:17.376340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce0623-1c21-497f-a955-bc992c473c30 admin admin] PATH_INFO: `/v3/users/59b99e1c26964900aef55f28407463a3` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:17.377612 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce0623-1c21-497f-a955-bc992c473c30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59b99e1c26964900aef55f28407463a3)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:17.379615 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce0623-1c21-497f-a955-bc992c473c30 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:17.618080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0fce0623-1c21-497f-a955-bc992c473c30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59b99e1c26964900aef55f28407463a3'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:17.677499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0fce0623-1c21-497f-a955-bc992c473c30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59b99e1c26964900aef55f28407463a3'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:17.678116 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0fce0623-1c21-497f-a955-bc992c473c30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59b99e1c26964900aef55f28407463a3'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:17.678719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0fce0623-1c21-497f-a955-bc992c473c30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59b99e1c26964900aef55f28407463a3'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:17.687220 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0fce0623-1c21-497f-a955-bc992c473c30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59b99e1c26964900aef55f28407463a3'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:17.705098 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1209/2420] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:17 2022] DELETE /identity/v3/users/59b99e1c26964900aef55f28407463a3 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:18.163862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cc8ff-0b66-4b8e-a998-834ad82772d1 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:18.165523 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-583cc8ff-0b66-4b8e-a998-834ad82772d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:18.179994 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cc8ff-0b66-4b8e-a998-834ad82772d1 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:18.242629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea4a36-2c09-4bf9-83aa-9f5b3ba99f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:18.243527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea4a36-2c09-4bf9-83aa-9f5b3ba99f35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:18.243816 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea4a36-2c09-4bf9-83aa-9f5b3ba99f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:18.244114 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea4a36-2c09-4bf9-83aa-9f5b3ba99f35 admin admin] PATH_INFO: `/v3/projects/66699060047d4f2f904e871c3c5ff873` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:18.245378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea4a36-2c09-4bf9-83aa-9f5b3ba99f35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66699060047d4f2f904e871c3c5ff873)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:18.247379 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea4a36-2c09-4bf9-83aa-9f5b3ba99f35 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:18.438501 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7dea4a36-2c09-4bf9-83aa-9f5b3ba99f35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66699060047d4f2f904e871c3c5ff873', 'request_id': 'req-7dea4a36-2c09-4bf9-83aa-9f5b3ba99f35'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:18.443879 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7dea4a36-2c09-4bf9-83aa-9f5b3ba99f35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66699060047d4f2f904e871c3c5ff873', 'request_id': 'req-7dea4a36-2c09-4bf9-83aa-9f5b3ba99f35'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:18.450231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7dea4a36-2c09-4bf9-83aa-9f5b3ba99f35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66699060047d4f2f904e871c3c5ff873', 'request_id': 'req-7dea4a36-2c09-4bf9-83aa-9f5b3ba99f35'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:18.558933 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7dea4a36-2c09-4bf9-83aa-9f5b3ba99f35 admin admin] The token cache is being invalidate because project 66699060047d4f2f904e871c3c5ff873 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:18.559306 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7dea4a36-2c09-4bf9-83aa-9f5b3ba99f35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:18.560060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7dea4a36-2c09-4bf9-83aa-9f5b3ba99f35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:18.561859 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1212/2421] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:18 2022] DELETE /identity/v3/projects/66699060047d4f2f904e871c3c5ff873 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:18.566843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfe460-e59f-4b37-81f9-86411601a25b None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:18.567171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfe460-e59f-4b37-81f9-86411601a25b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:18.567448 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bfe460-e59f-4b37-81f9-86411601a25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:18.586062 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-62bfe460-e59f-4b37-81f9-86411601a25b None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:18.587410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-62bfe460-e59f-4b37-81f9-86411601a25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:18.665731 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1210/2422] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:48:18.673739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf0a05-53b5-4d90-8786-49c69477a90e None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:18.674029 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf0a05-53b5-4d90-8786-49c69477a90e None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:18.674635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf0a05-53b5-4d90-8786-49c69477a90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:18.700688 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-67cf0a05-53b5-4d90-8786-49c69477a90e None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:18.701815 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-67cf0a05-53b5-4d90-8786-49c69477a90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:18.711455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1213/2423] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:18.718605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfe460-e59f-4b37-81f9-86411601a25b None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:18.725819 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bfe460-e59f-4b37-81f9-86411601a25b None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:18.730431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f0ca5-d1b0-4b12-a79b-d53027e9e290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:18.731118 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f0ca5-d1b0-4b12-a79b-d53027e9e290 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:18.731396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f0ca5-d1b0-4b12-a79b-d53027e9e290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:18.731669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f0ca5-d1b0-4b12-a79b-d53027e9e290 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:18.732395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f0ca5-d1b0-4b12-a79b-d53027e9e290 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:48:18.732699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f0ca5-d1b0-4b12-a79b-d53027e9e290 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:18.735023 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f0ca5-d1b0-4b12-a79b-d53027e9e290 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:18.747246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1211/2424] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 15:48:18.752270 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf0a05-53b5-4d90-8786-49c69477a90e None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:18.757875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:18.762542 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf0a05-53b5-4d90-8786-49c69477a90e None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:18.767162 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e2a21-4154-4509-bcec-8fa9dbf36cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:18.767816 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e2a21-4154-4509-bcec-8fa9dbf36cbf admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:18.768097 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e2a21-4154-4509-bcec-8fa9dbf36cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:18.768488 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91e2a21-4154-4509-bcec-8fa9dbf36cbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:18.769194 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e2a21-4154-4509-bcec-8fa9dbf36cbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:18.771120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91e2a21-4154-4509-bcec-8fa9dbf36cbf admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:18.881483 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1214/2425] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:18 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:18.882071 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f0ca5-d1b0-4b12-a79b-d53027e9e290 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:18.895785 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f0ca5-d1b0-4b12-a79b-d53027e9e290 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:18.933654 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:18.980748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a24bc7-9268-4fae-9f9b-9181170acc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:18.981426 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a24bc7-9268-4fae-9f9b-9181170acc34 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:18.981731 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a24bc7-9268-4fae-9f9b-9181170acc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:18.982005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a24bc7-9268-4fae-9f9b-9181170acc34 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:18.982689 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a24bc7-9268-4fae-9f9b-9181170acc34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:18.986257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a24bc7-9268-4fae-9f9b-9181170acc34 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:19.080255 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1212/2426] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:18 2022] POST /identity/v3/users => generated 291 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:19.084651 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e2a21-4154-4509-bcec-8fa9dbf36cbf admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:19.096165 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91e2a21-4154-4509-bcec-8fa9dbf36cbf admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:19.101480 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1472fb8-5e1d-484c-9062-c1ca8b067797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:19.102158 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1472fb8-5e1d-484c-9062-c1ca8b067797 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:19.102444 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1472fb8-5e1d-484c-9062-c1ca8b067797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:19.102725 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1472fb8-5e1d-484c-9062-c1ca8b067797 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:19.103502 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1472fb8-5e1d-484c-9062-c1ca8b067797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:19.105269 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1472fb8-5e1d-484c-9062-c1ca8b067797 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:19.119832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1215/2427] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:19.121233 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a24bc7-9268-4fae-9f9b-9181170acc34 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:19.132363 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a24bc7-9268-4fae-9f9b-9181170acc34 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:19.137058 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1150330-514d-4f04-9136-c9d4481da5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:19.138165 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1150330-514d-4f04-9136-c9d4481da5a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:19.138165 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1150330-514d-4f04-9136-c9d4481da5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:19.138439 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1150330-514d-4f04-9136-c9d4481da5a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:19.139198 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1150330-514d-4f04-9136-c9d4481da5a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:19.140951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1150330-514d-4f04-9136-c9d4481da5a0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:19.152541 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1213/2428] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:19.155885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1472fb8-5e1d-484c-9062-c1ca8b067797 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:19.166635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1472fb8-5e1d-484c-9062-c1ca8b067797 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:19.172379 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb83bfd-fa10-45ca-a293-d1dabdb11db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:19.173269 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb83bfd-fa10-45ca-a293-d1dabdb11db5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:19.173553 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb83bfd-fa10-45ca-a293-d1dabdb11db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:19.173830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb83bfd-fa10-45ca-a293-d1dabdb11db5 admin admin] PATH_INFO: `/v3/projects/5c6ac53cfe6843cab7e3b116547b9308/users/2bfccd7c8e4b4ca8bcba4fa9443aa075/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:19.189851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb83bfd-fa10-45ca-a293-d1dabdb11db5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c6ac53cfe6843cab7e3b116547b9308, user_id=2bfccd7c8e4b4ca8bcba4fa9443aa075, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:19.192639 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb83bfd-fa10-45ca-a293-d1dabdb11db5 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:19.358240 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1216/2429] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:19 2022] PUT /identity/v3/projects/5c6ac53cfe6843cab7e3b116547b9308/users/2bfccd7c8e4b4ca8bcba4fa9443aa075/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:19.370549 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae596029-3dde-436d-bf4e-15a91b513d5b None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:19.370911 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae596029-3dde-436d-bf4e-15a91b513d5b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:19.371202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae596029-3dde-436d-bf4e-15a91b513d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:19.402030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ae596029-3dde-436d-bf4e-15a91b513d5b None None] MFA Rules not processed for user `2bfccd7c8e4b4ca8bcba4fa9443aa075`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:19.403306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ae596029-3dde-436d-bf4e-15a91b513d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:19.476000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1214/2430] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:19.535880 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb83bfd-fa10-45ca-a293-d1dabdb11db5 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:19.598996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb83bfd-fa10-45ca-a293-d1dabdb11db5 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:19.603797 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42b289-b9b2-4f63-b1cd-56d17a902136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:19.604396 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42b289-b9b2-4f63-b1cd-56d17a902136 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:19.604680 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42b289-b9b2-4f63-b1cd-56d17a902136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:19.604957 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42b289-b9b2-4f63-b1cd-56d17a902136 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:19.605683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42b289-b9b2-4f63-b1cd-56d17a902136 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:19.607642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42b289-b9b2-4f63-b1cd-56d17a902136 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:19.708084 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1217/2431] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:19 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:19.712172 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae596029-3dde-436d-bf4e-15a91b513d5b None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:19.723562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae596029-3dde-436d-bf4e-15a91b513d5b None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:19.778075 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293174ff-3712-4681-827a-bd92bb8a6cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:19.778760 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293174ff-3712-4681-827a-bd92bb8a6cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:19.779074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293174ff-3712-4681-827a-bd92bb8a6cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:19.779343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293174ff-3712-4681-827a-bd92bb8a6cf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:19.780038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293174ff-3712-4681-827a-bd92bb8a6cf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:19.782992 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293174ff-3712-4681-827a-bd92bb8a6cf5 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:19.893251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1215/2432] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:19 2022] POST /identity/v3/users => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:19.894472 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42b289-b9b2-4f63-b1cd-56d17a902136 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:19.907170 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42b289-b9b2-4f63-b1cd-56d17a902136 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:19.911926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a96a9-d7e2-41db-bdd5-2bfea7bd7a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:19.912594 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a96a9-d7e2-41db-bdd5-2bfea7bd7a3e admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:19.912875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a96a9-d7e2-41db-bdd5-2bfea7bd7a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:19.913148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a96a9-d7e2-41db-bdd5-2bfea7bd7a3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:19.913870 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a96a9-d7e2-41db-bdd5-2bfea7bd7a3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:19.915541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a96a9-d7e2-41db-bdd5-2bfea7bd7a3e admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:19.926918 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1218/2433] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:48:19.931121 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293174ff-3712-4681-827a-bd92bb8a6cf5 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:19.941652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293174ff-3712-4681-827a-bd92bb8a6cf5 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:19.946222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccdb39-d35c-4b1d-9663-28ffe46b3d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:19.947130 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccdb39-d35c-4b1d-9663-28ffe46b3d26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:19.947408 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccdb39-d35c-4b1d-9663-28ffe46b3d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:19.947685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccdb39-d35c-4b1d-9663-28ffe46b3d26 admin admin] PATH_INFO: `/v3/projects/c632b85a659641a5b455b50d60a16e93/users/242e34d2bfbe487193f9afa4d5f625dd/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:19.963379 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dccdb39-d35c-4b1d-9663-28ffe46b3d26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c632b85a659641a5b455b50d60a16e93, user_id=242e34d2bfbe487193f9afa4d5f625dd, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:19.966018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dccdb39-d35c-4b1d-9663-28ffe46b3d26 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:20.072598 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1216/2434] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:19 2022] PUT /identity/v3/projects/c632b85a659641a5b455b50d60a16e93/users/242e34d2bfbe487193f9afa4d5f625dd/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:20.076796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a96a9-d7e2-41db-bdd5-2bfea7bd7a3e admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:20.087951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a96a9-d7e2-41db-bdd5-2bfea7bd7a3e admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:20.152181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968b7e5-e47c-4d3c-b758-6dfd4790dbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:20.152931 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968b7e5-e47c-4d3c-b758-6dfd4790dbd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:20.153376 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968b7e5-e47c-4d3c-b758-6dfd4790dbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:20.153508 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968b7e5-e47c-4d3c-b758-6dfd4790dbd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:20.154198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968b7e5-e47c-4d3c-b758-6dfd4790dbd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:20.155855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968b7e5-e47c-4d3c-b758-6dfd4790dbd6 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:20.168154 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1219/2435] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:20.170638 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccdb39-d35c-4b1d-9663-28ffe46b3d26 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:20.176467 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:20.181900 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccdb39-d35c-4b1d-9663-28ffe46b3d26 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:20.187346 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba512074-46e3-4776-91f5-1bdf656b8c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:20.187977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba512074-46e3-4776-91f5-1bdf656b8c35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:20.188257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba512074-46e3-4776-91f5-1bdf656b8c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:20.188531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba512074-46e3-4776-91f5-1bdf656b8c35 admin admin] PATH_INFO: `/v3/projects/c632b85a659641a5b455b50d60a16e93/users/242e34d2bfbe487193f9afa4d5f625dd/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:20.190644 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba512074-46e3-4776-91f5-1bdf656b8c35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c632b85a659641a5b455b50d60a16e93, user_id=242e34d2bfbe487193f9afa4d5f625dd, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:20.193452 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba512074-46e3-4776-91f5-1bdf656b8c35 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:20.205083 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1217/2436] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:20 2022] PUT /identity/v3/projects/c632b85a659641a5b455b50d60a16e93/users/242e34d2bfbe487193f9afa4d5f625dd/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:20.215290 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fbe87-0af5-4338-88a0-b393468b59d7 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:20.215664 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fbe87-0af5-4338-88a0-b393468b59d7 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:20.215949 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fbe87-0af5-4338-88a0-b393468b59d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:20.247145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-cf4fbe87-0af5-4338-88a0-b393468b59d7 None None] MFA Rules not processed for user `242e34d2bfbe487193f9afa4d5f625dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:20.247145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-cf4fbe87-0af5-4338-88a0-b393468b59d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:20.252281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:20.316786 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1220/2437] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:20.329000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba512074-46e3-4776-91f5-1bdf656b8c35 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:20.330692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ba512074-46e3-4776-91f5-1bdf656b8c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:20.388590 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba512074-46e3-4776-91f5-1bdf656b8c35 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:20.392951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce399e0-096f-4fee-a196-fb785ee7305b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:20.393599 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce399e0-096f-4fee-a196-fb785ee7305b service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:20.393883 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce399e0-096f-4fee-a196-fb785ee7305b service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:20.394161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce399e0-096f-4fee-a196-fb785ee7305b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:20.403580 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce399e0-096f-4fee-a196-fb785ee7305b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:20.404982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce399e0-096f-4fee-a196-fb785ee7305b service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:20.415194 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1218/2438] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:48:20.594119 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fbe87-0af5-4338-88a0-b393468b59d7 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:20.603826 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fbe87-0af5-4338-88a0-b393468b59d7 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:20.607556 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4689099-7300-4308-9a5c-014a9ab6ae3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:20.608196 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4689099-7300-4308-9a5c-014a9ab6ae3e service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:20.609645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4689099-7300-4308-9a5c-014a9ab6ae3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:20.609645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4689099-7300-4308-9a5c-014a9ab6ae3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:20.667965 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4689099-7300-4308-9a5c-014a9ab6ae3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:20.669323 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4689099-7300-4308-9a5c-014a9ab6ae3e service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:20.679104 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1221/2439] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:48:21.184000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce399e0-096f-4fee-a196-fb785ee7305b service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:21.249795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce399e0-096f-4fee-a196-fb785ee7305b service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:21.254354 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e51b1-78ad-4497-a74c-651112f8d8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:21.255257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e51b1-78ad-4497-a74c-651112f8d8fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:21.255540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e51b1-78ad-4497-a74c-651112f8d8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:21.255813 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e51b1-78ad-4497-a74c-651112f8d8fb admin admin] PATH_INFO: `/v3/users/2bfccd7c8e4b4ca8bcba4fa9443aa075` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:21.256986 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e51b1-78ad-4497-a74c-651112f8d8fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bfccd7c8e4b4ca8bcba4fa9443aa075)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:21.258906 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e51b1-78ad-4497-a74c-651112f8d8fb admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:21.367058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4689099-7300-4308-9a5c-014a9ab6ae3e service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:21.367406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c4689099-7300-4308-9a5c-014a9ab6ae3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:21.425393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4689099-7300-4308-9a5c-014a9ab6ae3e service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:21.429378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6dfdf7-d824-4e0f-a96f-35f7e7babda6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:21.433044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dfdf7-d824-4e0f-a96f-35f7e7babda6 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:21.433044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dfdf7-d824-4e0f-a96f-35f7e7babda6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:21.433044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dfdf7-d824-4e0f-a96f-35f7e7babda6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:21.433044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5b6dfdf7-d824-4e0f-a96f-35f7e7babda6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:21.444290 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6dfdf7-d824-4e0f-a96f-35f7e7babda6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:21.445716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6dfdf7-d824-4e0f-a96f-35f7e7babda6 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:21.456006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1222/2440] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:48:21.498224 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ed2e51b1-78ad-4497-a74c-651112f8d8fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bfccd7c8e4b4ca8bcba4fa9443aa075'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:21.564131 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ed2e51b1-78ad-4497-a74c-651112f8d8fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bfccd7c8e4b4ca8bcba4fa9443aa075'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:21.564791 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ed2e51b1-78ad-4497-a74c-651112f8d8fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bfccd7c8e4b4ca8bcba4fa9443aa075'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:21.565415 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ed2e51b1-78ad-4497-a74c-651112f8d8fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bfccd7c8e4b4ca8bcba4fa9443aa075'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:21.572748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ed2e51b1-78ad-4497-a74c-651112f8d8fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bfccd7c8e4b4ca8bcba4fa9443aa075'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:21.590536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1219/2441] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:21 2022] DELETE /identity/v3/users/2bfccd7c8e4b4ca8bcba4fa9443aa075 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:21.595334 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e69135-b197-4ad0-a89d-ed98ffd65741 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:21.595850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e69135-b197-4ad0-a89d-ed98ffd65741 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:21.596026 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e69135-b197-4ad0-a89d-ed98ffd65741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:21.613644 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-89e69135-b197-4ad0-a89d-ed98ffd65741 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:21.614877 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-89e69135-b197-4ad0-a89d-ed98ffd65741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:21.657273 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:21.676456 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1223/2442] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:21.686190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e51b1-78ad-4497-a74c-651112f8d8fb admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:21.687653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ed2e51b1-78ad-4497-a74c-651112f8d8fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:21.701932 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:21.747506 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e51b1-78ad-4497-a74c-651112f8d8fb admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:21.751214 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ecc96-bf0f-476c-a94d-e7c25b824454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:21.751864 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ecc96-bf0f-476c-a94d-e7c25b824454 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:21.752148 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ecc96-bf0f-476c-a94d-e7c25b824454 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:21.752425 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ecc96-bf0f-476c-a94d-e7c25b824454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:21.762912 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ecc96-bf0f-476c-a94d-e7c25b824454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:21.764451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ecc96-bf0f-476c-a94d-e7c25b824454 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:21.774880 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1220/2443] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:48:22.329120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e69135-b197-4ad0-a89d-ed98ffd65741 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:22.330608 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-89e69135-b197-4ad0-a89d-ed98ffd65741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:22.349692 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e69135-b197-4ad0-a89d-ed98ffd65741 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:22.356958 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ee1b8-6771-42e2-ad07-65a0e559490a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:22.358328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ee1b8-6771-42e2-ad07-65a0e559490a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:22.358328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ee1b8-6771-42e2-ad07-65a0e559490a admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:22.358813 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ee1b8-6771-42e2-ad07-65a0e559490a admin admin] PATH_INFO: `/v3/projects/5c6ac53cfe6843cab7e3b116547b9308` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:22.360435 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ee1b8-6771-42e2-ad07-65a0e559490a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c6ac53cfe6843cab7e3b116547b9308)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:22.370776 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ee1b8-6771-42e2-ad07-65a0e559490a admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:22.472312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-6f7ee1b8-6771-42e2-ad07-65a0e559490a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c6ac53cfe6843cab7e3b116547b9308', 'request_id': 'req-6f7ee1b8-6771-42e2-ad07-65a0e559490a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:22.478217 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-6f7ee1b8-6771-42e2-ad07-65a0e559490a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c6ac53cfe6843cab7e3b116547b9308', 'request_id': 'req-6f7ee1b8-6771-42e2-ad07-65a0e559490a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:22.485113 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-6f7ee1b8-6771-42e2-ad07-65a0e559490a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c6ac53cfe6843cab7e3b116547b9308', 'request_id': 'req-6f7ee1b8-6771-42e2-ad07-65a0e559490a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:22.573051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-6f7ee1b8-6771-42e2-ad07-65a0e559490a admin admin] The token cache is being invalidate because project 5c6ac53cfe6843cab7e3b116547b9308 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:22.576291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-6f7ee1b8-6771-42e2-ad07-65a0e559490a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:22.576291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-6f7ee1b8-6771-42e2-ad07-65a0e559490a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:22.576291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1224/2444] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:22 2022] DELETE /identity/v3/projects/5c6ac53cfe6843cab7e3b116547b9308 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:22.580666 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ecc96-bf0f-476c-a94d-e7c25b824454 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:22.591239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ecc96-bf0f-476c-a94d-e7c25b824454 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:22.664073 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6f75d-2a3a-41ab-93b3-f65d93eb1b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:22.665200 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6f75d-2a3a-41ab-93b3-f65d93eb1b65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:22.665694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6f75d-2a3a-41ab-93b3-f65d93eb1b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:22.666181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6f75d-2a3a-41ab-93b3-f65d93eb1b65 admin admin] PATH_INFO: `/v3/users/242e34d2bfbe487193f9afa4d5f625dd` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:22.667635 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca6f75d-2a3a-41ab-93b3-f65d93eb1b65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=242e34d2bfbe487193f9afa4d5f625dd)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:22.674672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca6f75d-2a3a-41ab-93b3-f65d93eb1b65 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:22.870357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5ca6f75d-2a3a-41ab-93b3-f65d93eb1b65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '242e34d2bfbe487193f9afa4d5f625dd'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:22.940353 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5ca6f75d-2a3a-41ab-93b3-f65d93eb1b65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '242e34d2bfbe487193f9afa4d5f625dd'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:22.941036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5ca6f75d-2a3a-41ab-93b3-f65d93eb1b65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '242e34d2bfbe487193f9afa4d5f625dd'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:22.941711 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5ca6f75d-2a3a-41ab-93b3-f65d93eb1b65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '242e34d2bfbe487193f9afa4d5f625dd'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:22.949365 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5ca6f75d-2a3a-41ab-93b3-f65d93eb1b65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '242e34d2bfbe487193f9afa4d5f625dd'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:22.973685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1221/2445] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:22 2022] DELETE /identity/v3/users/242e34d2bfbe487193f9afa4d5f625dd => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:23.441789 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ee1b8-6771-42e2-ad07-65a0e559490a admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:23.443498 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-6f7ee1b8-6771-42e2-ad07-65a0e559490a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:23.449387 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6f75d-2a3a-41ab-93b3-f65d93eb1b65 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:23.454441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5ca6f75d-2a3a-41ab-93b3-f65d93eb1b65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:23.461798 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ee1b8-6771-42e2-ad07-65a0e559490a admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:23.505892 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:23.535198 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6f75d-2a3a-41ab-93b3-f65d93eb1b65 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:23.539941 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019cd070-08cb-427a-8938-822603586083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:23.540779 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cd070-08cb-427a-8938-822603586083 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:23.541059 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cd070-08cb-427a-8938-822603586083 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:23.541446 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d92a5-c172-4625-a42d-5c3c5412c08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:23.541598 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cd070-08cb-427a-8938-822603586083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:23.542733 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d92a5-c172-4625-a42d-5c3c5412c08f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:23.543582 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d92a5-c172-4625-a42d-5c3c5412c08f admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:23.543726 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d92a5-c172-4625-a42d-5c3c5412c08f admin admin] PATH_INFO: `/v3/projects/c632b85a659641a5b455b50d60a16e93` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:23.543832 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-019cd070-08cb-427a-8938-822603586083 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:23.545261 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d92a5-c172-4625-a42d-5c3c5412c08f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c632b85a659641a5b455b50d60a16e93)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:23.547934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d92a5-c172-4625-a42d-5c3c5412c08f admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:23.559865 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019cd070-08cb-427a-8938-822603586083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:23.561129 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019cd070-08cb-427a-8938-822603586083 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:23.577450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1222/2446] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:48:23.671558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e66d92a5-c172-4625-a42d-5c3c5412c08f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c632b85a659641a5b455b50d60a16e93', 'request_id': 'req-e66d92a5-c172-4625-a42d-5c3c5412c08f'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:23.681696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e66d92a5-c172-4625-a42d-5c3c5412c08f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c632b85a659641a5b455b50d60a16e93', 'request_id': 'req-e66d92a5-c172-4625-a42d-5c3c5412c08f'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:23.694715 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e66d92a5-c172-4625-a42d-5c3c5412c08f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c632b85a659641a5b455b50d60a16e93', 'request_id': 'req-e66d92a5-c172-4625-a42d-5c3c5412c08f'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:23.762407 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:23.768231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e66d92a5-c172-4625-a42d-5c3c5412c08f admin admin] The token cache is being invalidate because project c632b85a659641a5b455b50d60a16e93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:23.774452 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e66d92a5-c172-4625-a42d-5c3c5412c08f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:23.774452 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e66d92a5-c172-4625-a42d-5c3c5412c08f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:23.774452 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1225/2447] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:23 2022] DELETE /identity/v3/projects/c632b85a659641a5b455b50d60a16e93 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:23.776135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145a754-03f1-43f6-bdf7-f1133ac2d84f None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:23.776614 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145a754-03f1-43f6-bdf7-f1133ac2d84f None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:23.776719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145a754-03f1-43f6-bdf7-f1133ac2d84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:23.795884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-4145a754-03f1-43f6-bdf7-f1133ac2d84f None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:23.800286 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4145a754-03f1-43f6-bdf7-f1133ac2d84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:23.859995 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1223/2448] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:48:23.868276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca88275-3a7b-486b-a837-3e62b31c0be7 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:23.868560 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca88275-3a7b-486b-a837-3e62b31c0be7 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:23.868831 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca88275-3a7b-486b-a837-3e62b31c0be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:23.888760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-5ca88275-3a7b-486b-a837-3e62b31c0be7 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:23.890132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5ca88275-3a7b-486b-a837-3e62b31c0be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:23.898404 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1226/2449] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:23.905954 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145a754-03f1-43f6-bdf7-f1133ac2d84f None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:23.916935 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145a754-03f1-43f6-bdf7-f1133ac2d84f None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:23.922662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c33c1-f10b-47fa-ad79-84057a443f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:23.923396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c33c1-f10b-47fa-ad79-84057a443f8f admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:23.923676 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c33c1-f10b-47fa-ad79-84057a443f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:23.923950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c33c1-f10b-47fa-ad79-84057a443f8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:23.924690 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c33c1-f10b-47fa-ad79-84057a443f8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:48:23.924994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c33c1-f10b-47fa-ad79-84057a443f8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:23.926607 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c33c1-f10b-47fa-ad79-84057a443f8f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:23.943226 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1224/2450] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:48:23.944269 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca88275-3a7b-486b-a837-3e62b31c0be7 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:23.954630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca88275-3a7b-486b-a837-3e62b31c0be7 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:23.959472 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89339d02-d61e-4340-8d39-79dcb5f47ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:23.964549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89339d02-d61e-4340-8d39-79dcb5f47ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:23.964549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89339d02-d61e-4340-8d39-79dcb5f47ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:23.964549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89339d02-d61e-4340-8d39-79dcb5f47ff6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:23.964549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89339d02-d61e-4340-8d39-79dcb5f47ff6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:23.964549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89339d02-d61e-4340-8d39-79dcb5f47ff6 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:24.057510 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1227/2451] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:23 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:48:24.062603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c33c1-f10b-47fa-ad79-84057a443f8f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:24.073211 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c33c1-f10b-47fa-ad79-84057a443f8f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:24.142524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931800d-107d-4e27-8ec4-feedde8a884a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:24.143243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931800d-107d-4e27-8ec4-feedde8a884a admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:24.143529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931800d-107d-4e27-8ec4-feedde8a884a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:24.143805 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931800d-107d-4e27-8ec4-feedde8a884a admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:24.144652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931800d-107d-4e27-8ec4-feedde8a884a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:24.146803 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931800d-107d-4e27-8ec4-feedde8a884a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:24.227615 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1225/2452] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:24 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:48:24.231509 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89339d02-d61e-4340-8d39-79dcb5f47ff6 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:24.242456 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89339d02-d61e-4340-8d39-79dcb5f47ff6 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:24.247175 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8697f73-8ab8-4d7f-a5f7-9be90b77071e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:24.247767 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8697f73-8ab8-4d7f-a5f7-9be90b77071e admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:24.248042 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8697f73-8ab8-4d7f-a5f7-9be90b77071e admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:24.248315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8697f73-8ab8-4d7f-a5f7-9be90b77071e admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:24.249039 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8697f73-8ab8-4d7f-a5f7-9be90b77071e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:24.251075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8697f73-8ab8-4d7f-a5f7-9be90b77071e admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:24.261292 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1228/2453] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:24.266246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931800d-107d-4e27-8ec4-feedde8a884a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:24.276418 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931800d-107d-4e27-8ec4-feedde8a884a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:24.283567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cae57-e30a-4109-9129-1ecac7d1cb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:24.290537 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cae57-e30a-4109-9129-1ecac7d1cb84 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:24.290537 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cae57-e30a-4109-9129-1ecac7d1cb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:24.290537 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cae57-e30a-4109-9129-1ecac7d1cb84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:24.290537 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3cae57-e30a-4109-9129-1ecac7d1cb84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:24.290537 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3cae57-e30a-4109-9129-1ecac7d1cb84 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:24.299576 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1226/2454] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:48:24.303678 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8697f73-8ab8-4d7f-a5f7-9be90b77071e admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:24.314401 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8697f73-8ab8-4d7f-a5f7-9be90b77071e admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:24.319193 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d06d4-39a6-491a-8527-b9287897ee68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:24.320085 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d06d4-39a6-491a-8527-b9287897ee68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:24.320395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d06d4-39a6-491a-8527-b9287897ee68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:24.320667 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d06d4-39a6-491a-8527-b9287897ee68 admin admin] PATH_INFO: `/v3/projects/d5231086b3c0494d890eb665eab8d303/users/4cca7cfbb1be4d92a603a715af2728f8/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:24.340499 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d06d4-39a6-491a-8527-b9287897ee68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5231086b3c0494d890eb665eab8d303, user_id=4cca7cfbb1be4d92a603a715af2728f8, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:24.344158 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d06d4-39a6-491a-8527-b9287897ee68 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:24.449879 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1229/2455] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:24 2022] PUT /identity/v3/projects/d5231086b3c0494d890eb665eab8d303/users/4cca7cfbb1be4d92a603a715af2728f8/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:24.469595 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff058bec-89ce-46c4-b7fe-ce99591775da None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:24.469884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff058bec-89ce-46c4-b7fe-ce99591775da None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:24.470322 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff058bec-89ce-46c4-b7fe-ce99591775da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:24.502940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ff058bec-89ce-46c4-b7fe-ce99591775da None None] MFA Rules not processed for user `4cca7cfbb1be4d92a603a715af2728f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:24.504255 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ff058bec-89ce-46c4-b7fe-ce99591775da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:24.538525 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:24.574014 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1227/2456] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:24.585338 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d06d4-39a6-491a-8527-b9287897ee68 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:24.587256 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b26d06d4-39a6-491a-8527-b9287897ee68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:24.649024 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d06d4-39a6-491a-8527-b9287897ee68 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:24.652856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81c397-eee6-4caf-b8f0-3ebf59a15253 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:24.653614 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81c397-eee6-4caf-b8f0-3ebf59a15253 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:24.653878 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81c397-eee6-4caf-b8f0-3ebf59a15253 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:24.654174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81c397-eee6-4caf-b8f0-3ebf59a15253 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:24.663286 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81c397-eee6-4caf-b8f0-3ebf59a15253 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:24.664806 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81c397-eee6-4caf-b8f0-3ebf59a15253 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:24.673831 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1230/2457] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:48:26.092104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff058bec-89ce-46c4-b7fe-ce99591775da None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:26.095749 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ff058bec-89ce-46c4-b7fe-ce99591775da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:26.176956 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff058bec-89ce-46c4-b7fe-ce99591775da None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:26.181851 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32580ae4-1c06-4356-baa1-2c6e5b184f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:26.183138 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32580ae4-1c06-4356-baa1-2c6e5b184f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:26.183138 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32580ae4-1c06-4356-baa1-2c6e5b184f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:26.183380 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32580ae4-1c06-4356-baa1-2c6e5b184f5d admin admin] PATH_INFO: `/v3/users/9ee0106af0894acb911831433ca22caf` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:26.184662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32580ae4-1c06-4356-baa1-2c6e5b184f5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ee0106af0894acb911831433ca22caf)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:26.186645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32580ae4-1c06-4356-baa1-2c6e5b184f5d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:26.390473 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-32580ae4-1c06-4356-baa1-2c6e5b184f5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ee0106af0894acb911831433ca22caf'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:26.442655 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-32580ae4-1c06-4356-baa1-2c6e5b184f5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ee0106af0894acb911831433ca22caf'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:26.443270 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-32580ae4-1c06-4356-baa1-2c6e5b184f5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ee0106af0894acb911831433ca22caf'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:26.443861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-32580ae4-1c06-4356-baa1-2c6e5b184f5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ee0106af0894acb911831433ca22caf'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:26.451027 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-32580ae4-1c06-4356-baa1-2c6e5b184f5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ee0106af0894acb911831433ca22caf'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:26.472628 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1228/2458] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:26 2022] DELETE /identity/v3/users/9ee0106af0894acb911831433ca22caf => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:26.596143 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7451e-0276-4e35-82ce-a39618780f4c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:26.597099 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7451e-0276-4e35-82ce-a39618780f4c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:26.597469 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7451e-0276-4e35-82ce-a39618780f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:26.616318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-06f7451e-0276-4e35-82ce-a39618780f4c None None] MFA Rules not processed for user `4cca7cfbb1be4d92a603a715af2728f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:26.617569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-06f7451e-0276-4e35-82ce-a39618780f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:26.683470 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1231/2459] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:26.749989 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32580ae4-1c06-4356-baa1-2c6e5b184f5d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:26.749989 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-32580ae4-1c06-4356-baa1-2c6e5b184f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:26.824528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:26.843064 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32580ae4-1c06-4356-baa1-2c6e5b184f5d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:26.847942 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba67ef-8bb6-48d7-8659-9cd500e2c8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:26.848819 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba67ef-8bb6-48d7-8659-9cd500e2c8f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:26.849253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba67ef-8bb6-48d7-8659-9cd500e2c8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:26.849708 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba67ef-8bb6-48d7-8659-9cd500e2c8f4 admin admin] PATH_INFO: `/v3/projects/2f6efc6860644128973230d227ad005a` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:26.850609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba67ef-8bb6-48d7-8659-9cd500e2c8f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f6efc6860644128973230d227ad005a)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:26.852833 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba67ef-8bb6-48d7-8659-9cd500e2c8f4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:26.962523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-adba67ef-8bb6-48d7-8659-9cd500e2c8f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f6efc6860644128973230d227ad005a', 'request_id': 'req-adba67ef-8bb6-48d7-8659-9cd500e2c8f4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:26.970713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-adba67ef-8bb6-48d7-8659-9cd500e2c8f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f6efc6860644128973230d227ad005a', 'request_id': 'req-adba67ef-8bb6-48d7-8659-9cd500e2c8f4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:26.977705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-adba67ef-8bb6-48d7-8659-9cd500e2c8f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f6efc6860644128973230d227ad005a', 'request_id': 'req-adba67ef-8bb6-48d7-8659-9cd500e2c8f4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:27.049581 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-adba67ef-8bb6-48d7-8659-9cd500e2c8f4 admin admin] The token cache is being invalidate because project 2f6efc6860644128973230d227ad005a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:27.049861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-adba67ef-8bb6-48d7-8659-9cd500e2c8f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:27.050694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-adba67ef-8bb6-48d7-8659-9cd500e2c8f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:27.052612 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1229/2460] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:26 2022] DELETE /identity/v3/projects/2f6efc6860644128973230d227ad005a => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:27.060365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5ae95-212b-4ab9-80c1-7f1497357274 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:27.060365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5ae95-212b-4ab9-80c1-7f1497357274 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:27.060365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5ae95-212b-4ab9-80c1-7f1497357274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:27.081226 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-97b5ae95-212b-4ab9-80c1-7f1497357274 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:27.082552 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-97b5ae95-212b-4ab9-80c1-7f1497357274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:27.128344 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:27.158153 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1232/2461] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:27.166707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e8d8c-cf8b-4508-93b8-bef45826ddb0 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:27.168752 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e8d8c-cf8b-4508-93b8-bef45826ddb0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:27.168752 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e8d8c-cf8b-4508-93b8-bef45826ddb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:27.205302 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-005e8d8c-cf8b-4508-93b8-bef45826ddb0 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:27.206677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-005e8d8c-cf8b-4508-93b8-bef45826ddb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:27.220233 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1230/2462] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:48:27.224241 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5ae95-212b-4ab9-80c1-7f1497357274 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:27.246622 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5ae95-212b-4ab9-80c1-7f1497357274 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:27.270388 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849d902-d771-441e-90b7-cf943c7b50ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:27.270388 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849d902-d771-441e-90b7-cf943c7b50ed admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:27.270388 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849d902-d771-441e-90b7-cf943c7b50ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:27.272396 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849d902-d771-441e-90b7-cf943c7b50ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:27.272396 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849d902-d771-441e-90b7-cf943c7b50ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:48:27.272396 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849d902-d771-441e-90b7-cf943c7b50ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:27.276745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849d902-d771-441e-90b7-cf943c7b50ed admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:27.304365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1233/2463] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 15:48:27.304589 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e8d8c-cf8b-4508-93b8-bef45826ddb0 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:27.319610 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e8d8c-cf8b-4508-93b8-bef45826ddb0 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:27.326428 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7e6b5-142c-4284-83f7-5b9d8cb9318e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:27.328078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7e6b5-142c-4284-83f7-5b9d8cb9318e admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:27.328467 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7e6b5-142c-4284-83f7-5b9d8cb9318e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:27.328816 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7e6b5-142c-4284-83f7-5b9d8cb9318e admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:27.329652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7e6b5-142c-4284-83f7-5b9d8cb9318e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:27.331874 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7e6b5-142c-4284-83f7-5b9d8cb9318e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:27.444774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1231/2464] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:27 2022] POST /identity/v3/projects => generated 367 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:27.445086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849d902-d771-441e-90b7-cf943c7b50ed admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:27.460453 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849d902-d771-441e-90b7-cf943c7b50ed admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:27.564146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7816fc-49da-4bcd-9ab7-024e3dad2d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:27.565356 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7816fc-49da-4bcd-9ab7-024e3dad2d2e admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:27.566187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7816fc-49da-4bcd-9ab7-024e3dad2d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:27.566571 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7816fc-49da-4bcd-9ab7-024e3dad2d2e admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:27.568345 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7816fc-49da-4bcd-9ab7-024e3dad2d2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:27.571576 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7816fc-49da-4bcd-9ab7-024e3dad2d2e admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:27.712590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1234/2465] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:27 2022] POST /identity/v3/users => generated 284 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:48:27.716628 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7e6b5-142c-4284-83f7-5b9d8cb9318e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:27.728682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7e6b5-142c-4284-83f7-5b9d8cb9318e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:27.734233 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa35c6f-ad04-49b6-a90b-fed160774f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:27.738546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa35c6f-ad04-49b6-a90b-fed160774f96 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:27.738546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa35c6f-ad04-49b6-a90b-fed160774f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:27.738546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa35c6f-ad04-49b6-a90b-fed160774f96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:27.738546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa35c6f-ad04-49b6-a90b-fed160774f96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:27.738546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa35c6f-ad04-49b6-a90b-fed160774f96 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:27.758432 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1232/2466] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:27.762485 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7816fc-49da-4bcd-9ab7-024e3dad2d2e admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:27.777418 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7816fc-49da-4bcd-9ab7-024e3dad2d2e admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:27.789127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f600a6-cedc-4799-863c-d29c69da5f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:27.789925 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f600a6-cedc-4799-863c-d29c69da5f00 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:27.790306 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f600a6-cedc-4799-863c-d29c69da5f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:27.798249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f600a6-cedc-4799-863c-d29c69da5f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:27.798249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f600a6-cedc-4799-863c-d29c69da5f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:27.798249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f600a6-cedc-4799-863c-d29c69da5f00 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:27.805591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1235/2467] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:27.810055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa35c6f-ad04-49b6-a90b-fed160774f96 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:27.823505 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa35c6f-ad04-49b6-a90b-fed160774f96 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:27.829698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f945a9a-715a-4779-b08d-3bd277bfaaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:27.830604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f945a9a-715a-4779-b08d-3bd277bfaaf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:27.831083 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f945a9a-715a-4779-b08d-3bd277bfaaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:27.831367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f945a9a-715a-4779-b08d-3bd277bfaaf6 admin admin] PATH_INFO: `/v3/projects/6553f684fe924cc2bc8b01b1c29db9f0/users/e4216253f56c42488959a35a96e5df80/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:27.851941 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f945a9a-715a-4779-b08d-3bd277bfaaf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6553f684fe924cc2bc8b01b1c29db9f0, user_id=e4216253f56c42488959a35a96e5df80, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:27.854977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f945a9a-715a-4779-b08d-3bd277bfaaf6 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:27.931246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1233/2468] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:27 2022] PUT /identity/v3/projects/6553f684fe924cc2bc8b01b1c29db9f0/users/e4216253f56c42488959a35a96e5df80/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:27.944231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d471ef4-011f-4bfc-89a0-07335035f6a4 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:27.944624 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d471ef4-011f-4bfc-89a0-07335035f6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:27.944834 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d471ef4-011f-4bfc-89a0-07335035f6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:27.976027 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-7d471ef4-011f-4bfc-89a0-07335035f6a4 None None] MFA Rules not processed for user `e4216253f56c42488959a35a96e5df80`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:27.983314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7d471ef4-011f-4bfc-89a0-07335035f6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:28.047328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1236/2469] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:27 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:28.051838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f945a9a-715a-4779-b08d-3bd277bfaaf6 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:28.115798 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f945a9a-715a-4779-b08d-3bd277bfaaf6 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:28.121050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680bd8f-4e2d-4479-879a-cfabe7af8fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:28.121780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680bd8f-4e2d-4479-879a-cfabe7af8fba admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:28.126157 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d471ef4-011f-4bfc-89a0-07335035f6a4 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:28.126739 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680bd8f-4e2d-4479-879a-cfabe7af8fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:28.127074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680bd8f-4e2d-4479-879a-cfabe7af8fba admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:28.127858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680bd8f-4e2d-4479-879a-cfabe7af8fba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:28.128454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7d471ef4-011f-4bfc-89a0-07335035f6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:28.129867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680bd8f-4e2d-4479-879a-cfabe7af8fba admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:28.147541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d471ef4-011f-4bfc-89a0-07335035f6a4 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:28.152856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbd0be-158d-4316-a826-f6a3fe437206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:28.153741 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbd0be-158d-4316-a826-f6a3fe437206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:28.154021 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbd0be-158d-4316-a826-f6a3fe437206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:28.154295 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbd0be-158d-4316-a826-f6a3fe437206 admin admin] PATH_INFO: `/v3/users/4cadd4bc4d36437484f495d0e49d9881` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:28.156545 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adbd0be-158d-4316-a826-f6a3fe437206 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cadd4bc4d36437484f495d0e49d9881)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:28.158459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adbd0be-158d-4316-a826-f6a3fe437206 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:28.242904 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1234/2470] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:28 2022] POST /identity/v3/projects => generated 367 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:28.247026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680bd8f-4e2d-4479-879a-cfabe7af8fba admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:28.251246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:28.310256 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680bd8f-4e2d-4479-879a-cfabe7af8fba admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:28.315688 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f68ca-2b0d-474b-a847-d89deb18f42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:28.316440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f68ca-2b0d-474b-a847-d89deb18f42c admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:28.317071 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f68ca-2b0d-474b-a847-d89deb18f42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:28.317655 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f68ca-2b0d-474b-a847-d89deb18f42c admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:28.319275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f68ca-2b0d-474b-a847-d89deb18f42c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:28.319843 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:28.322058 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f68ca-2b0d-474b-a847-d89deb18f42c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:28.333724 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7adbd0be-158d-4316-a826-f6a3fe437206 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cadd4bc4d36437484f495d0e49d9881'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:28.425524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7adbd0be-158d-4316-a826-f6a3fe437206 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cadd4bc4d36437484f495d0e49d9881'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:28.432357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1235/2471] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:28 2022] POST /identity/v3/users => generated 290 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:48:28.435420 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7adbd0be-158d-4316-a826-f6a3fe437206 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cadd4bc4d36437484f495d0e49d9881'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:28.437023 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f68ca-2b0d-474b-a847-d89deb18f42c admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:28.447140 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f68ca-2b0d-474b-a847-d89deb18f42c admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:28.450823 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7adbd0be-158d-4316-a826-f6a3fe437206 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cadd4bc4d36437484f495d0e49d9881'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:28.452418 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7adbd0be-158d-4316-a826-f6a3fe437206 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cadd4bc4d36437484f495d0e49d9881'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:28.452808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b46c93-73d8-4782-a976-c8f0d5040d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:28.455917 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b46c93-73d8-4782-a976-c8f0d5040d42 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:28.455917 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b46c93-73d8-4782-a976-c8f0d5040d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:28.455917 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b46c93-73d8-4782-a976-c8f0d5040d42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:28.455917 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b46c93-73d8-4782-a976-c8f0d5040d42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:28.456345 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b46c93-73d8-4782-a976-c8f0d5040d42 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:28.459098 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1237/2472] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:28 2022] DELETE /identity/v3/users/4cadd4bc4d36437484f495d0e49d9881 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:28.470118 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1236/2473] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:48:28.471261 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbd0be-158d-4316-a826-f6a3fe437206 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:28.472661 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7adbd0be-158d-4316-a826-f6a3fe437206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:28.542593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbd0be-158d-4316-a826-f6a3fe437206 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:28.548145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e3883-bc59-40e6-92a6-5f895dd16700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:28.551103 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e3883-bc59-40e6-92a6-5f895dd16700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:28.551103 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e3883-bc59-40e6-92a6-5f895dd16700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:28.551103 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e3883-bc59-40e6-92a6-5f895dd16700 admin admin] PATH_INFO: `/v3/projects/3f5c7d966c5b40c19a4d0b5be85d8e88/users/59d4f32615054f5186f3aa950fb11ea1/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:28.567008 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e3883-bc59-40e6-92a6-5f895dd16700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f5c7d966c5b40c19a4d0b5be85d8e88, user_id=59d4f32615054f5186f3aa950fb11ea1, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:28.570191 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e3883-bc59-40e6-92a6-5f895dd16700 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:28.684079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1238/2474] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:28 2022] PUT /identity/v3/projects/3f5c7d966c5b40c19a4d0b5be85d8e88/users/59d4f32615054f5186f3aa950fb11ea1/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:28.688742 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b46c93-73d8-4782-a976-c8f0d5040d42 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:28.698544 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b46c93-73d8-4782-a976-c8f0d5040d42 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:28.736220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e3883-bc59-40e6-92a6-5f895dd16700 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:28.738147 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-2f8e3883-bc59-40e6-92a6-5f895dd16700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:28.764859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab74ca-b291-42d2-846c-5656c580d77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:28.765649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab74ca-b291-42d2-846c-5656c580d77b admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:28.766012 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab74ca-b291-42d2-846c-5656c580d77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:28.766367 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab74ca-b291-42d2-846c-5656c580d77b admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:28.767571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab74ca-b291-42d2-846c-5656c580d77b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:28.769870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab74ca-b291-42d2-846c-5656c580d77b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:28.785664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1237/2475] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:28 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:28.786648 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab74ca-b291-42d2-846c-5656c580d77b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:28.797024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab74ca-b291-42d2-846c-5656c580d77b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:28.801808 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68f131-2c5f-49a1-b1b6-3e1e2b31a1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:28.802608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68f131-2c5f-49a1-b1b6-3e1e2b31a1af admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:28.802913 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68f131-2c5f-49a1-b1b6-3e1e2b31a1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:28.803198 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68f131-2c5f-49a1-b1b6-3e1e2b31a1af admin admin] PATH_INFO: `/v3/projects/3f5c7d966c5b40c19a4d0b5be85d8e88/users/59d4f32615054f5186f3aa950fb11ea1/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:28.805257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd68f131-2c5f-49a1-b1b6-3e1e2b31a1af admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f5c7d966c5b40c19a4d0b5be85d8e88, user_id=59d4f32615054f5186f3aa950fb11ea1, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:28.808111 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd68f131-2c5f-49a1-b1b6-3e1e2b31a1af admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:28.813712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e3883-bc59-40e6-92a6-5f895dd16700 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:28.819953 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1238/2476] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:28 2022] PUT /identity/v3/projects/3f5c7d966c5b40c19a4d0b5be85d8e88/users/59d4f32615054f5186f3aa950fb11ea1/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:28.830154 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7c422-e3aa-4258-828e-22b971d2ad4b None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:28.830589 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7c422-e3aa-4258-828e-22b971d2ad4b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:28.830851 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7c422-e3aa-4258-828e-22b971d2ad4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:28.853273 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:28.859795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-96b7c422-e3aa-4258-828e-22b971d2ad4b None None] MFA Rules not processed for user `59d4f32615054f5186f3aa950fb11ea1`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:28.861028 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-96b7c422-e3aa-4258-828e-22b971d2ad4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:28.871749 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cce6d3-2565-4efe-ab15-87fe34838f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:28.872603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cce6d3-2565-4efe-ab15-87fe34838f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:28.872888 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cce6d3-2565-4efe-ab15-87fe34838f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:28.873199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cce6d3-2565-4efe-ab15-87fe34838f7c admin admin] PATH_INFO: `/v3/projects/fa53787b85ab4869a4bb67f68b3dde86` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:28.875071 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cce6d3-2565-4efe-ab15-87fe34838f7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa53787b85ab4869a4bb67f68b3dde86)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:28.876990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cce6d3-2565-4efe-ab15-87fe34838f7c admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:28.929844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:28.952135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1239/2477] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:28 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:28.963686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7c422-e3aa-4258-828e-22b971d2ad4b None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:28.965238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-96b7c422-e3aa-4258-828e-22b971d2ad4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:29.018488 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d6cce6d3-2565-4efe-ab15-87fe34838f7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa53787b85ab4869a4bb67f68b3dde86', 'request_id': 'req-d6cce6d3-2565-4efe-ab15-87fe34838f7c'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:29.025571 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d6cce6d3-2565-4efe-ab15-87fe34838f7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa53787b85ab4869a4bb67f68b3dde86', 'request_id': 'req-d6cce6d3-2565-4efe-ab15-87fe34838f7c'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:29.038479 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d6cce6d3-2565-4efe-ab15-87fe34838f7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa53787b85ab4869a4bb67f68b3dde86', 'request_id': 'req-d6cce6d3-2565-4efe-ab15-87fe34838f7c'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:29.040994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7c422-e3aa-4258-828e-22b971d2ad4b None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:29.093233 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc333ec-e96d-4bea-99bd-e1688ec7ea38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:29.093992 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc333ec-e96d-4bea-99bd-e1688ec7ea38 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:29.094274 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc333ec-e96d-4bea-99bd-e1688ec7ea38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:29.094553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc333ec-e96d-4bea-99bd-e1688ec7ea38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:29.112173 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d6cce6d3-2565-4efe-ab15-87fe34838f7c admin admin] The token cache is being invalidate because project fa53787b85ab4869a4bb67f68b3dde86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:29.112534 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d6cce6d3-2565-4efe-ab15-87fe34838f7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:29.113485 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d6cce6d3-2565-4efe-ab15-87fe34838f7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:29.115182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1239/2478] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:28 2022] DELETE /identity/v3/projects/fa53787b85ab4869a4bb67f68b3dde86 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:29.120215 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc47e8c-bb92-441c-ba61-f59929e1618d None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:29.120606 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc47e8c-bb92-441c-ba61-f59929e1618d None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:29.121094 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc47e8c-bb92-441c-ba61-f59929e1618d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:29.140049 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-abc47e8c-bb92-441c-ba61-f59929e1618d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:29.141388 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-abc47e8c-bb92-441c-ba61-f59929e1618d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:29.158437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc333ec-e96d-4bea-99bd-e1688ec7ea38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:29.159920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc333ec-e96d-4bea-99bd-e1688ec7ea38 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:29.169805 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1240/2479] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:28 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:48:29.210310 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1240/2480] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:48:29.218950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad763c57-660d-4d9f-ae04-ccec7e82ad92 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:29.219246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad763c57-660d-4d9f-ae04-ccec7e82ad92 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:29.219551 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad763c57-660d-4d9f-ae04-ccec7e82ad92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:29.238208 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ad763c57-660d-4d9f-ae04-ccec7e82ad92 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:29.239454 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ad763c57-660d-4d9f-ae04-ccec7e82ad92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:29.247680 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1241/2481] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:29.252026 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc47e8c-bb92-441c-ba61-f59929e1618d None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:29.261682 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc47e8c-bb92-441c-ba61-f59929e1618d None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:29.267675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff050109-0321-435e-b3d1-878a817ca81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:29.277662 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff050109-0321-435e-b3d1-878a817ca81e admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:29.277662 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff050109-0321-435e-b3d1-878a817ca81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:29.277662 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff050109-0321-435e-b3d1-878a817ca81e admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:29.277662 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff050109-0321-435e-b3d1-878a817ca81e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:48:29.277662 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff050109-0321-435e-b3d1-878a817ca81e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:29.277662 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff050109-0321-435e-b3d1-878a817ca81e admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:29.286063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1241/2482] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:48:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:48:29.290392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad763c57-660d-4d9f-ae04-ccec7e82ad92 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:29.303214 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad763c57-660d-4d9f-ae04-ccec7e82ad92 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:29.312056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30832b88-2002-4cbb-a027-58e05f74dd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:29.312056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30832b88-2002-4cbb-a027-58e05f74dd05 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:29.312056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30832b88-2002-4cbb-a027-58e05f74dd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:29.312056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30832b88-2002-4cbb-a027-58e05f74dd05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:29.312652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30832b88-2002-4cbb-a027-58e05f74dd05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:29.312652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30832b88-2002-4cbb-a027-58e05f74dd05 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:29.414647 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1242/2483] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:29 2022] POST /identity/v3/projects => generated 365 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:48:29.415727 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff050109-0321-435e-b3d1-878a817ca81e admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:29.426551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff050109-0321-435e-b3d1-878a817ca81e admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:29.478980 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:29.485568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395fd48-edfc-4676-b050-e8eac931e5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:29.486263 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395fd48-edfc-4676-b050-e8eac931e5ac admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:29.486544 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395fd48-edfc-4676-b050-e8eac931e5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:29.486826 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395fd48-edfc-4676-b050-e8eac931e5ac admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:29.487577 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395fd48-edfc-4676-b050-e8eac931e5ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:29.489503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7395fd48-edfc-4676-b050-e8eac931e5ac admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:29.593212 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1242/2484] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:29 2022] POST /identity/v3/users => generated 283 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:48:29.596635 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30832b88-2002-4cbb-a027-58e05f74dd05 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:29.607797 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30832b88-2002-4cbb-a027-58e05f74dd05 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:29.612764 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825eedc-2842-4b92-8627-e71f590318b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:29.613389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825eedc-2842-4b92-8627-e71f590318b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:29.613669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825eedc-2842-4b92-8627-e71f590318b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:29.613961 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825eedc-2842-4b92-8627-e71f590318b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:29.614693 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9825eedc-2842-4b92-8627-e71f590318b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:29.616376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9825eedc-2842-4b92-8627-e71f590318b9 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:29.626604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1243/2485] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:29.630729 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395fd48-edfc-4676-b050-e8eac931e5ac admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:29.641344 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395fd48-edfc-4676-b050-e8eac931e5ac admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:29.646196 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fa117-07d1-4016-aaba-8259d8c95514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:29.646863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fa117-07d1-4016-aaba-8259d8c95514 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:29.647363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fa117-07d1-4016-aaba-8259d8c95514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:29.648060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fa117-07d1-4016-aaba-8259d8c95514 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:29.648816 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835fa117-07d1-4016-aaba-8259d8c95514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:29.650518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835fa117-07d1-4016-aaba-8259d8c95514 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:29.661657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1243/2486] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:29.665784 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825eedc-2842-4b92-8627-e71f590318b9 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:29.676312 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825eedc-2842-4b92-8627-e71f590318b9 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:29.681212 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10822f2f-cdc1-45ca-812f-b3ba7b0fff5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:29.685441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10822f2f-cdc1-45ca-812f-b3ba7b0fff5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:29.685441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10822f2f-cdc1-45ca-812f-b3ba7b0fff5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:29.685441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10822f2f-cdc1-45ca-812f-b3ba7b0fff5c admin admin] PATH_INFO: `/v3/projects/d637aa134f4d4e49861ba4f8649ca8d8/users/b6e5c6fbf5244a31817c5bdb3f2b4764/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:29.695104 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:29.698284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10822f2f-cdc1-45ca-812f-b3ba7b0fff5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d637aa134f4d4e49861ba4f8649ca8d8, user_id=b6e5c6fbf5244a31817c5bdb3f2b4764, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:29.700957 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10822f2f-cdc1-45ca-812f-b3ba7b0fff5c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:29.792065 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1244/2487] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:29 2022] PUT /identity/v3/projects/d637aa134f4d4e49861ba4f8649ca8d8/users/b6e5c6fbf5244a31817c5bdb3f2b4764/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:29.803832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c976e-5623-4332-8814-e81c723230f5 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:29.804397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c976e-5623-4332-8814-e81c723230f5 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:29.804397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c976e-5623-4332-8814-e81c723230f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:29.835493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-452c976e-5623-4332-8814-e81c723230f5 None None] MFA Rules not processed for user `b6e5c6fbf5244a31817c5bdb3f2b4764`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:29.837025 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-452c976e-5623-4332-8814-e81c723230f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:29.903887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1244/2488] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:29 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:29.908898 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10822f2f-cdc1-45ca-812f-b3ba7b0fff5c admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:29.973450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10822f2f-cdc1-45ca-812f-b3ba7b0fff5c admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:29.978437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb980c-8018-472d-99ee-c0b8fd1c6008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:29.978997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb980c-8018-472d-99ee-c0b8fd1c6008 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:29.979411 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb980c-8018-472d-99ee-c0b8fd1c6008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:29.979588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb980c-8018-472d-99ee-c0b8fd1c6008 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:29.980253 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb980c-8018-472d-99ee-c0b8fd1c6008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:29.982174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb980c-8018-472d-99ee-c0b8fd1c6008 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:30.059184 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1245/2489] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:29 2022] POST /identity/v3/projects => generated 363 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:30.061600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c976e-5623-4332-8814-e81c723230f5 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:30.073487 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c976e-5623-4332-8814-e81c723230f5 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:30.153152 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f78dc-04cd-4cc5-9d09-e610b04e1592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:30.153809 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f78dc-04cd-4cc5-9d09-e610b04e1592 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:30.154167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f78dc-04cd-4cc5-9d09-e610b04e1592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:30.154521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f78dc-04cd-4cc5-9d09-e610b04e1592 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:30.155370 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f78dc-04cd-4cc5-9d09-e610b04e1592 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:30.158527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f78dc-04cd-4cc5-9d09-e610b04e1592 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:30.214555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1245/2490] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:30 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:48:30.219321 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb980c-8018-472d-99ee-c0b8fd1c6008 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:30.242174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb980c-8018-472d-99ee-c0b8fd1c6008 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:30.246275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66da4b1-fc03-4dba-9f2f-c33acf0fd2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:30.247106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66da4b1-fc03-4dba-9f2f-c33acf0fd2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:30.247496 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66da4b1-fc03-4dba-9f2f-c33acf0fd2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:30.248031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66da4b1-fc03-4dba-9f2f-c33acf0fd2b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:30.249062 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66da4b1-fc03-4dba-9f2f-c33acf0fd2b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:30.250816 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66da4b1-fc03-4dba-9f2f-c33acf0fd2b8 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:30.272038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1246/2491] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:30.276684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f78dc-04cd-4cc5-9d09-e610b04e1592 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:30.288152 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f78dc-04cd-4cc5-9d09-e610b04e1592 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:30.293981 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f71c8-6614-4be9-8a7c-3e32c487c8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:30.294998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f71c8-6614-4be9-8a7c-3e32c487c8fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:30.295364 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f71c8-6614-4be9-8a7c-3e32c487c8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:30.295742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f71c8-6614-4be9-8a7c-3e32c487c8fd admin admin] PATH_INFO: `/v3/projects/d34ffb4f71c142b991375e02a7030694/users/550c8de5c73a4378abfa50ac411953a2/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:30.318401 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f71c8-6614-4be9-8a7c-3e32c487c8fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d34ffb4f71c142b991375e02a7030694, user_id=550c8de5c73a4378abfa50ac411953a2, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:30.321013 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f71c8-6614-4be9-8a7c-3e32c487c8fd admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:30.433433 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1246/2492] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:30 2022] PUT /identity/v3/projects/d34ffb4f71c142b991375e02a7030694/users/550c8de5c73a4378abfa50ac411953a2/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:30.437135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66da4b1-fc03-4dba-9f2f-c33acf0fd2b8 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:30.450313 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66da4b1-fc03-4dba-9f2f-c33acf0fd2b8 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:30.526438 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87b67d-0b6b-46ff-97ae-2780b0ba65a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:30.526438 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87b67d-0b6b-46ff-97ae-2780b0ba65a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:30.526438 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87b67d-0b6b-46ff-97ae-2780b0ba65a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:30.526438 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87b67d-0b6b-46ff-97ae-2780b0ba65a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:30.526856 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87b67d-0b6b-46ff-97ae-2780b0ba65a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:30.527857 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87b67d-0b6b-46ff-97ae-2780b0ba65a8 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:30.542849 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1247/2493] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:48:30.544752 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f71c8-6614-4be9-8a7c-3e32c487c8fd admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:30.555990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f71c8-6614-4be9-8a7c-3e32c487c8fd admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:30.560905 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b445fa-0766-4222-a3c2-664ade8b1c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:30.562868 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b445fa-0766-4222-a3c2-664ade8b1c62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:30.563180 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b445fa-0766-4222-a3c2-664ade8b1c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:30.563454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b445fa-0766-4222-a3c2-664ade8b1c62 admin admin] PATH_INFO: `/v3/projects/d34ffb4f71c142b991375e02a7030694/users/550c8de5c73a4378abfa50ac411953a2/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:30.565461 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b445fa-0766-4222-a3c2-664ade8b1c62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d34ffb4f71c142b991375e02a7030694, user_id=550c8de5c73a4378abfa50ac411953a2, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:30.569909 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b445fa-0766-4222-a3c2-664ade8b1c62 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:30.581584 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1247/2494] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:30 2022] PUT /identity/v3/projects/d34ffb4f71c142b991375e02a7030694/users/550c8de5c73a4378abfa50ac411953a2/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:30.591909 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a3947-a703-4214-8cac-8be1a583af94 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:30.592103 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a3947-a703-4214-8cac-8be1a583af94 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:30.592441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a3947-a703-4214-8cac-8be1a583af94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:30.620751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ad9a3947-a703-4214-8cac-8be1a583af94 None None] MFA Rules not processed for user `550c8de5c73a4378abfa50ac411953a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:30.621968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ad9a3947-a703-4214-8cac-8be1a583af94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:30.674586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:30.705488 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1248/2495] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:30 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:48:30.713163 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b445fa-0766-4222-a3c2-664ade8b1c62 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:30.714837 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-25b445fa-0766-4222-a3c2-664ade8b1c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:30.735667 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:30.774511 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b445fa-0766-4222-a3c2-664ade8b1c62 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:30.778292 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded312b-3a53-4713-9697-dea7ca44a025 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:30.779011 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded312b-3a53-4713-9697-dea7ca44a025 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:30.782548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded312b-3a53-4713-9697-dea7ca44a025 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:30.782548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded312b-3a53-4713-9697-dea7ca44a025 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:30.798328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ded312b-3a53-4713-9697-dea7ca44a025 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:30.800403 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ded312b-3a53-4713-9697-dea7ca44a025 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:30.811599 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1248/2496] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:30 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:48:31.065096 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a3947-a703-4214-8cac-8be1a583af94 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:31.075636 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a3947-a703-4214-8cac-8be1a583af94 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:31.080436 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e1d9d-5c60-404e-b05b-cb553b377b04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:31.081202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e1d9d-5c60-404e-b05b-cb553b377b04 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:31.084030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e1d9d-5c60-404e-b05b-cb553b377b04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:31.084818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e1d9d-5c60-404e-b05b-cb553b377b04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:31.167922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e1d9d-5c60-404e-b05b-cb553b377b04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:31.171349 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e1d9d-5c60-404e-b05b-cb553b377b04 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:31.193477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1249/2497] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:31 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:48:31.390013 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded312b-3a53-4713-9697-dea7ca44a025 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:31.391459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-2ded312b-3a53-4713-9697-dea7ca44a025 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:31.453134 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded312b-3a53-4713-9697-dea7ca44a025 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:31.457188 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea66c73-5c8b-42f1-aba0-94991b492198 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:31.461995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea66c73-5c8b-42f1-aba0-94991b492198 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:31.461995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea66c73-5c8b-42f1-aba0-94991b492198 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:31.461995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea66c73-5c8b-42f1-aba0-94991b492198 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:31.461995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5ea66c73-5c8b-42f1-aba0-94991b492198 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:31.471456 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea66c73-5c8b-42f1-aba0-94991b492198 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:31.472893 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea66c73-5c8b-42f1-aba0-94991b492198 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:31.481705 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1249/2498] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:48:32.443973 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e1d9d-5c60-404e-b05b-cb553b377b04 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:32.445682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-4d6e1d9d-5c60-404e-b05b-cb553b377b04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:32.519687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e1d9d-5c60-404e-b05b-cb553b377b04 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:32.524497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de9dc3-badd-4447-959c-e79811322762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:32.525387 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de9dc3-badd-4447-959c-e79811322762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:32.525667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de9dc3-badd-4447-959c-e79811322762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:32.525943 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de9dc3-badd-4447-959c-e79811322762 admin admin] PATH_INFO: `/v3/users/e4216253f56c42488959a35a96e5df80` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:32.527193 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de9dc3-badd-4447-959c-e79811322762 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4216253f56c42488959a35a96e5df80)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:32.529144 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de9dc3-badd-4447-959c-e79811322762 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:32.703587 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-29de9dc3-badd-4447-959c-e79811322762 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4216253f56c42488959a35a96e5df80'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:32.773012 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-29de9dc3-badd-4447-959c-e79811322762 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4216253f56c42488959a35a96e5df80'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:32.773633 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-29de9dc3-badd-4447-959c-e79811322762 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4216253f56c42488959a35a96e5df80'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:32.774236 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-29de9dc3-badd-4447-959c-e79811322762 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4216253f56c42488959a35a96e5df80'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:32.785454 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-29de9dc3-badd-4447-959c-e79811322762 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4216253f56c42488959a35a96e5df80'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:32.806500 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1250/2499] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:32 2022] DELETE /identity/v3/users/e4216253f56c42488959a35a96e5df80 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:32.811591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149117f7-cac0-460e-ab38-7ce7ce5d370d None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:32.812079 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149117f7-cac0-460e-ab38-7ce7ce5d370d None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:32.812202 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149117f7-cac0-460e-ab38-7ce7ce5d370d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:32.830562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-149117f7-cac0-460e-ab38-7ce7ce5d370d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:32.831989 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-149117f7-cac0-460e-ab38-7ce7ce5d370d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:32.915422 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1250/2500] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:32.929184 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de9dc3-badd-4447-959c-e79811322762 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:32.931799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-29de9dc3-badd-4447-959c-e79811322762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:32.940002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:33.000012 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de9dc3-badd-4447-959c-e79811322762 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:33.003641 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b0569-5858-4bfa-ba9d-493f71bbf0f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:33.004293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b0569-5858-4bfa-ba9d-493f71bbf0f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:33.004573 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b0569-5858-4bfa-ba9d-493f71bbf0f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:33.004847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b0569-5858-4bfa-ba9d-493f71bbf0f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:33.014870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b0569-5858-4bfa-ba9d-493f71bbf0f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:33.016267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b0569-5858-4bfa-ba9d-493f71bbf0f2 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:33.026073 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1251/2501] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:48:33.472775 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149117f7-cac0-460e-ab38-7ce7ce5d370d None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:33.474349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-149117f7-cac0-460e-ab38-7ce7ce5d370d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:33.487607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149117f7-cac0-460e-ab38-7ce7ce5d370d None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:33.492995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e83070-1a6c-4de7-85cb-2abdd09a045a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:33.493782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e83070-1a6c-4de7-85cb-2abdd09a045a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:33.494029 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e83070-1a6c-4de7-85cb-2abdd09a045a admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:33.494307 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e83070-1a6c-4de7-85cb-2abdd09a045a admin admin] PATH_INFO: `/v3/projects/6553f684fe924cc2bc8b01b1c29db9f0` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:33.495544 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e83070-1a6c-4de7-85cb-2abdd09a045a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6553f684fe924cc2bc8b01b1c29db9f0)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:33.501128 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e83070-1a6c-4de7-85cb-2abdd09a045a admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:33.631085 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-23e83070-1a6c-4de7-85cb-2abdd09a045a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6553f684fe924cc2bc8b01b1c29db9f0', 'request_id': 'req-23e83070-1a6c-4de7-85cb-2abdd09a045a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:33.641223 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-23e83070-1a6c-4de7-85cb-2abdd09a045a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6553f684fe924cc2bc8b01b1c29db9f0', 'request_id': 'req-23e83070-1a6c-4de7-85cb-2abdd09a045a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:33.642530 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:33.649002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-23e83070-1a6c-4de7-85cb-2abdd09a045a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6553f684fe924cc2bc8b01b1c29db9f0', 'request_id': 'req-23e83070-1a6c-4de7-85cb-2abdd09a045a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:33.744751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-23e83070-1a6c-4de7-85cb-2abdd09a045a admin admin] The token cache is being invalidate because project 6553f684fe924cc2bc8b01b1c29db9f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:33.745054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-23e83070-1a6c-4de7-85cb-2abdd09a045a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:33.745735 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-23e83070-1a6c-4de7-85cb-2abdd09a045a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:33.747336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1251/2502] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:33 2022] DELETE /identity/v3/projects/6553f684fe924cc2bc8b01b1c29db9f0 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:33.751853 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b0569-5858-4bfa-ba9d-493f71bbf0f2 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:33.767004 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b0569-5858-4bfa-ba9d-493f71bbf0f2 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:33.824503 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d5642-a41c-407e-bc91-2732e519444e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:33.825386 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d5642-a41c-407e-bc91-2732e519444e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:33.825667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d5642-a41c-407e-bc91-2732e519444e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:33.825943 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d5642-a41c-407e-bc91-2732e519444e admin admin] PATH_INFO: `/v3/users/59d4f32615054f5186f3aa950fb11ea1` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:33.827269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d5642-a41c-407e-bc91-2732e519444e admin admin] RBAC: Authorizing `identity:delete_user(user_id=59d4f32615054f5186f3aa950fb11ea1)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:33.829219 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d5642-a41c-407e-bc91-2732e519444e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:33.991488 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-583d5642-a41c-407e-bc91-2732e519444e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59d4f32615054f5186f3aa950fb11ea1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:34.071217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-583d5642-a41c-407e-bc91-2732e519444e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59d4f32615054f5186f3aa950fb11ea1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:34.071781 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-583d5642-a41c-407e-bc91-2732e519444e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59d4f32615054f5186f3aa950fb11ea1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:34.072380 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-583d5642-a41c-407e-bc91-2732e519444e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59d4f32615054f5186f3aa950fb11ea1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:34.080900 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-583d5642-a41c-407e-bc91-2732e519444e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59d4f32615054f5186f3aa950fb11ea1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:34.099399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1252/2503] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:33 2022] DELETE /identity/v3/users/59d4f32615054f5186f3aa950fb11ea1 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:34.490514 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e83070-1a6c-4de7-85cb-2abdd09a045a admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:34.492082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-23e83070-1a6c-4de7-85cb-2abdd09a045a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:34.506313 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e83070-1a6c-4de7-85cb-2abdd09a045a admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:34.562864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812b83f-ee12-48f0-ae1d-cef152456ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:34.563740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812b83f-ee12-48f0-ae1d-cef152456ee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:34.564025 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812b83f-ee12-48f0-ae1d-cef152456ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:34.564302 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812b83f-ee12-48f0-ae1d-cef152456ee1 admin admin] PATH_INFO: `/v3/projects/3f5c7d966c5b40c19a4d0b5be85d8e88` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:34.565512 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812b83f-ee12-48f0-ae1d-cef152456ee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f5c7d966c5b40c19a4d0b5be85d8e88)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:34.567467 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812b83f-ee12-48f0-ae1d-cef152456ee1 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:34.684273 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a812b83f-ee12-48f0-ae1d-cef152456ee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f5c7d966c5b40c19a4d0b5be85d8e88', 'request_id': 'req-a812b83f-ee12-48f0-ae1d-cef152456ee1'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:34.690979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a812b83f-ee12-48f0-ae1d-cef152456ee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f5c7d966c5b40c19a4d0b5be85d8e88', 'request_id': 'req-a812b83f-ee12-48f0-ae1d-cef152456ee1'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:34.698120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a812b83f-ee12-48f0-ae1d-cef152456ee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f5c7d966c5b40c19a4d0b5be85d8e88', 'request_id': 'req-a812b83f-ee12-48f0-ae1d-cef152456ee1'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:34.773373 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a812b83f-ee12-48f0-ae1d-cef152456ee1 admin admin] The token cache is being invalidate because project 3f5c7d966c5b40c19a4d0b5be85d8e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:34.773662 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a812b83f-ee12-48f0-ae1d-cef152456ee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:34.774287 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a812b83f-ee12-48f0-ae1d-cef152456ee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:34.775920 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1252/2504] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:34 2022] DELETE /identity/v3/projects/3f5c7d966c5b40c19a4d0b5be85d8e88 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:34.781140 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758c6fd-e656-4c62-ad0b-2c99572d51e0 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:34.781546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758c6fd-e656-4c62-ad0b-2c99572d51e0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:34.781788 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758c6fd-e656-4c62-ad0b-2c99572d51e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:34.801193 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-8758c6fd-e656-4c62-ad0b-2c99572d51e0 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:34.804384 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-8758c6fd-e656-4c62-ad0b-2c99572d51e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:34.824616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:34.864661 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1253/2505] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:34.872369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3b74-6f4e-42cf-8076-555e1528f181 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:34.872820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3b74-6f4e-42cf-8076-555e1528f181 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:34.873131 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f3b74-6f4e-42cf-8076-555e1528f181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:34.896013 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-1a9f3b74-6f4e-42cf-8076-555e1528f181 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:34.897390 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-1a9f3b74-6f4e-42cf-8076-555e1528f181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:34.908886 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1253/2506] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:34.913336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758c6fd-e656-4c62-ad0b-2c99572d51e0 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:34.924001 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758c6fd-e656-4c62-ad0b-2c99572d51e0 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:34.929113 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594af81-c676-4262-a506-99fcaad2b11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:34.929846 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594af81-c676-4262-a506-99fcaad2b11f admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:34.930231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594af81-c676-4262-a506-99fcaad2b11f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:34.930597 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594af81-c676-4262-a506-99fcaad2b11f admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:34.931480 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594af81-c676-4262-a506-99fcaad2b11f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:48:34.931856 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594af81-c676-4262-a506-99fcaad2b11f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:34.933570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594af81-c676-4262-a506-99fcaad2b11f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:34.948202 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1254/2507] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:48:34.965036 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f3b74-6f4e-42cf-8076-555e1528f181 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:34.974817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f3b74-6f4e-42cf-8076-555e1528f181 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:34.979689 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd930cf8-7150-4153-b480-51164d2bb369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:34.980345 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd930cf8-7150-4153-b480-51164d2bb369 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:34.980628 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd930cf8-7150-4153-b480-51164d2bb369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:34.980903 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd930cf8-7150-4153-b480-51164d2bb369 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:34.981590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd930cf8-7150-4153-b480-51164d2bb369 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:34.983534 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd930cf8-7150-4153-b480-51164d2bb369 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:35.074401 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1254/2508] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:34 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:35.079631 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594af81-c676-4262-a506-99fcaad2b11f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:35.092436 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594af81-c676-4262-a506-99fcaad2b11f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:35.169730 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c15ff-3268-4c50-a797-8cf7953827d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:35.170446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c15ff-3268-4c50-a797-8cf7953827d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:35.170841 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c15ff-3268-4c50-a797-8cf7953827d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:35.171144 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c15ff-3268-4c50-a797-8cf7953827d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:35.172822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c15ff-3268-4c50-a797-8cf7953827d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:35.175344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c15ff-3268-4c50-a797-8cf7953827d2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:35.282438 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1255/2509] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:35 2022] POST /identity/v3/users => generated 294 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:48:35.287115 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd930cf8-7150-4153-b480-51164d2bb369 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:35.298099 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd930cf8-7150-4153-b480-51164d2bb369 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:35.302677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a8c50-8c9c-4db2-9e79-be2d4df8546a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:35.303365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a8c50-8c9c-4db2-9e79-be2d4df8546a admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:35.303646 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a8c50-8c9c-4db2-9e79-be2d4df8546a admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:35.303919 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a8c50-8c9c-4db2-9e79-be2d4df8546a admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:35.304637 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98a8c50-8c9c-4db2-9e79-be2d4df8546a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:35.306235 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98a8c50-8c9c-4db2-9e79-be2d4df8546a admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:35.316603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1255/2510] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:35.320767 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c15ff-3268-4c50-a797-8cf7953827d2 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:35.332293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c15ff-3268-4c50-a797-8cf7953827d2 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:35.337080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb570c7-8f46-4f7b-81ec-2a046d2d0e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:35.337823 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb570c7-8f46-4f7b-81ec-2a046d2d0e25 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:35.338106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb570c7-8f46-4f7b-81ec-2a046d2d0e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:35.338377 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb570c7-8f46-4f7b-81ec-2a046d2d0e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:35.339268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb570c7-8f46-4f7b-81ec-2a046d2d0e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:35.340858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb570c7-8f46-4f7b-81ec-2a046d2d0e25 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:35.352544 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1256/2511] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:35.356728 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a8c50-8c9c-4db2-9e79-be2d4df8546a admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:35.371945 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a8c50-8c9c-4db2-9e79-be2d4df8546a admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:35.376870 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550aeba-509c-4114-84ee-6506c73a29aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:35.377756 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550aeba-509c-4114-84ee-6506c73a29aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:35.378034 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550aeba-509c-4114-84ee-6506c73a29aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:35.378359 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550aeba-509c-4114-84ee-6506c73a29aa admin admin] PATH_INFO: `/v3/projects/e4d6545231d145a297bb3aec14275033/users/da22090806e64812accd9fdda6c8139e/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:35.394680 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550aeba-509c-4114-84ee-6506c73a29aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4d6545231d145a297bb3aec14275033, user_id=da22090806e64812accd9fdda6c8139e, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:35.397509 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550aeba-509c-4114-84ee-6506c73a29aa admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:35.483000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1256/2512] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:35 2022] PUT /identity/v3/projects/e4d6545231d145a297bb3aec14275033/users/da22090806e64812accd9fdda6c8139e/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:35.494671 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12450f84-0f7c-4970-acbf-84299ff8c9c0 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:35.494987 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12450f84-0f7c-4970-acbf-84299ff8c9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:35.495264 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12450f84-0f7c-4970-acbf-84299ff8c9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:35.523239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-12450f84-0f7c-4970-acbf-84299ff8c9c0 None None] MFA Rules not processed for user `da22090806e64812accd9fdda6c8139e`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:35.524434 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-12450f84-0f7c-4970-acbf-84299ff8c9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:35.592077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1257/2513] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:35.596921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550aeba-509c-4114-84ee-6506c73a29aa admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:35.611517 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:35.660914 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550aeba-509c-4114-84ee-6506c73a29aa admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:35.666139 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff69be6-1f31-464b-83fe-1e9fb84cc616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:35.667679 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff69be6-1f31-464b-83fe-1e9fb84cc616 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:35.667817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff69be6-1f31-464b-83fe-1e9fb84cc616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:35.668291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff69be6-1f31-464b-83fe-1e9fb84cc616 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:35.668818 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff69be6-1f31-464b-83fe-1e9fb84cc616 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:35.671901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff69be6-1f31-464b-83fe-1e9fb84cc616 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:35.759380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1257/2514] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:35 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:35.764111 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12450f84-0f7c-4970-acbf-84299ff8c9c0 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:35.776304 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12450f84-0f7c-4970-acbf-84299ff8c9c0 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:35.795824 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:35.835105 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e4800-a71d-46d6-af4c-b6044744f99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:35.835787 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e4800-a71d-46d6-af4c-b6044744f99d admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:35.836072 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e4800-a71d-46d6-af4c-b6044744f99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:35.836347 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e4800-a71d-46d6-af4c-b6044744f99d admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:35.837048 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e4800-a71d-46d6-af4c-b6044744f99d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:35.839045 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e4800-a71d-46d6-af4c-b6044744f99d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:35.930709 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1258/2515] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:35 2022] POST /identity/v3/users => generated 299 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:35.931049 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff69be6-1f31-464b-83fe-1e9fb84cc616 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:35.938569 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff69be6-1f31-464b-83fe-1e9fb84cc616 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:35.943269 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a291f6-2ab2-4fbb-8f9f-7c40260194b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:35.943925 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a291f6-2ab2-4fbb-8f9f-7c40260194b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:35.944209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a291f6-2ab2-4fbb-8f9f-7c40260194b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:35.944481 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a291f6-2ab2-4fbb-8f9f-7c40260194b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:35.945196 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a291f6-2ab2-4fbb-8f9f-7c40260194b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:35.946780 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a291f6-2ab2-4fbb-8f9f-7c40260194b6 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:35.958330 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1258/2516] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:35.962696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e4800-a71d-46d6-af4c-b6044744f99d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:35.974077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e4800-a71d-46d6-af4c-b6044744f99d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:35.978745 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b68b26-8456-4a70-9eed-dca593481358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:35.979694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b68b26-8456-4a70-9eed-dca593481358 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:35.979985 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b68b26-8456-4a70-9eed-dca593481358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:35.980262 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b68b26-8456-4a70-9eed-dca593481358 admin admin] PATH_INFO: `/v3/projects/062c7dd4a5784c4abb5b12ec72ee75aa/users/f6744a895492446a8523a634dd174c88/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:35.997766 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b68b26-8456-4a70-9eed-dca593481358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=062c7dd4a5784c4abb5b12ec72ee75aa, user_id=f6744a895492446a8523a634dd174c88, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:36.001063 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b68b26-8456-4a70-9eed-dca593481358 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:36.072556 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1259/2517] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:35 2022] PUT /identity/v3/projects/062c7dd4a5784c4abb5b12ec72ee75aa/users/f6744a895492446a8523a634dd174c88/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:36.076723 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a291f6-2ab2-4fbb-8f9f-7c40260194b6 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:36.087854 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a291f6-2ab2-4fbb-8f9f-7c40260194b6 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:36.147285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8594df-46f0-47af-987b-200e726287b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:36.147991 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8594df-46f0-47af-987b-200e726287b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:36.148274 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8594df-46f0-47af-987b-200e726287b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:36.148551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8594df-46f0-47af-987b-200e726287b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:36.149269 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8594df-46f0-47af-987b-200e726287b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:36.150854 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8594df-46f0-47af-987b-200e726287b8 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:36.160884 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1259/2518] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:36.164991 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b68b26-8456-4a70-9eed-dca593481358 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:36.175700 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b68b26-8456-4a70-9eed-dca593481358 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:36.180175 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313de81-1a5c-49e5-8aef-0cfc049e021c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:36.181097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313de81-1a5c-49e5-8aef-0cfc049e021c admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:36.181317 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313de81-1a5c-49e5-8aef-0cfc049e021c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:36.181586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313de81-1a5c-49e5-8aef-0cfc049e021c admin admin] PATH_INFO: `/v3/projects/062c7dd4a5784c4abb5b12ec72ee75aa/users/f6744a895492446a8523a634dd174c88/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:36.183546 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1313de81-1a5c-49e5-8aef-0cfc049e021c admin admin] RBAC: Authorizing `identity:create_grant(project_id=062c7dd4a5784c4abb5b12ec72ee75aa, user_id=f6744a895492446a8523a634dd174c88, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:36.186115 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1313de81-1a5c-49e5-8aef-0cfc049e021c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:36.196684 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1260/2519] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:36 2022] PUT /identity/v3/projects/062c7dd4a5784c4abb5b12ec72ee75aa/users/f6744a895492446a8523a634dd174c88/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:36.206939 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c050a6-d336-4a0e-99fd-40f10782055a None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:36.207196 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c050a6-d336-4a0e-99fd-40f10782055a None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:36.207475 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c050a6-d336-4a0e-99fd-40f10782055a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:36.233337 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-62c050a6-d336-4a0e-99fd-40f10782055a None None] MFA Rules not processed for user `f6744a895492446a8523a634dd174c88`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:36.235073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-62c050a6-d336-4a0e-99fd-40f10782055a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:36.302718 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1260/2520] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:36.315379 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313de81-1a5c-49e5-8aef-0cfc049e021c admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:36.317193 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1313de81-1a5c-49e5-8aef-0cfc049e021c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:36.374475 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313de81-1a5c-49e5-8aef-0cfc049e021c admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:36.378170 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13cde5-5481-4609-900f-dc8df6e8c771 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:36.378810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13cde5-5481-4609-900f-dc8df6e8c771 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:36.379121 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13cde5-5481-4609-900f-dc8df6e8c771 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:36.379398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13cde5-5481-4609-900f-dc8df6e8c771 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:36.388342 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13cde5-5481-4609-900f-dc8df6e8c771 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:36.389771 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13cde5-5481-4609-900f-dc8df6e8c771 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:36.399704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1261/2521] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:48:36.464060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c050a6-d336-4a0e-99fd-40f10782055a None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:36.474536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:36.527832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c050a6-d336-4a0e-99fd-40f10782055a None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:36.532677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2d42b-8dce-4a23-9830-60f649fe9e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:36.533557 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2d42b-8dce-4a23-9830-60f649fe9e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:36.534035 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2d42b-8dce-4a23-9830-60f649fe9e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:36.534484 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2d42b-8dce-4a23-9830-60f649fe9e21 admin admin] PATH_INFO: `/v3/users/da22090806e64812accd9fdda6c8139e` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:36.536073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2d42b-8dce-4a23-9830-60f649fe9e21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da22090806e64812accd9fdda6c8139e)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:36.537990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2d42b-8dce-4a23-9830-60f649fe9e21 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:36.711583 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-62e2d42b-8dce-4a23-9830-60f649fe9e21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da22090806e64812accd9fdda6c8139e'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:36.783457 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-62e2d42b-8dce-4a23-9830-60f649fe9e21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da22090806e64812accd9fdda6c8139e'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:36.790615 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-62e2d42b-8dce-4a23-9830-60f649fe9e21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da22090806e64812accd9fdda6c8139e'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:36.802233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-62e2d42b-8dce-4a23-9830-60f649fe9e21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da22090806e64812accd9fdda6c8139e'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:36.802859 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-62e2d42b-8dce-4a23-9830-60f649fe9e21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da22090806e64812accd9fdda6c8139e'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:36.808908 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1261/2522] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:36 2022] DELETE /identity/v3/users/da22090806e64812accd9fdda6c8139e => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:36.813897 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ed2ce-befe-48fd-827f-f1e5453a8b1e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:36.814363 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ed2ce-befe-48fd-827f-f1e5453a8b1e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:36.814469 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ed2ce-befe-48fd-827f-f1e5453a8b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:36.832591 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-336ed2ce-befe-48fd-827f-f1e5453a8b1e None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:36.833843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-336ed2ce-befe-48fd-827f-f1e5453a8b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:36.894681 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1262/2523] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:48:36.905012 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2d42b-8dce-4a23-9830-60f649fe9e21 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:36.906700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-62e2d42b-8dce-4a23-9830-60f649fe9e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:36.965861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2d42b-8dce-4a23-9830-60f649fe9e21 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:36.969318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac465e2-fbb8-420b-9299-4aac829c3802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:36.969958 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac465e2-fbb8-420b-9299-4aac829c3802 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:36.970239 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac465e2-fbb8-420b-9299-4aac829c3802 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:36.970511 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac465e2-fbb8-420b-9299-4aac829c3802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:36.979546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac465e2-fbb8-420b-9299-4aac829c3802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:36.980953 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac465e2-fbb8-420b-9299-4aac829c3802 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:36.991248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1262/2524] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:48:37.473286 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ed2ce-befe-48fd-827f-f1e5453a8b1e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:37.474708 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-336ed2ce-befe-48fd-827f-f1e5453a8b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:37.488434 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ed2ce-befe-48fd-827f-f1e5453a8b1e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:37.493151 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532582bf-3406-4fa8-8db1-020102419716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:37.494046 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532582bf-3406-4fa8-8db1-020102419716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:37.494325 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532582bf-3406-4fa8-8db1-020102419716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:37.494594 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532582bf-3406-4fa8-8db1-020102419716 admin admin] PATH_INFO: `/v3/projects/e4d6545231d145a297bb3aec14275033` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:37.495811 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532582bf-3406-4fa8-8db1-020102419716 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4d6545231d145a297bb3aec14275033)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:37.497874 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532582bf-3406-4fa8-8db1-020102419716 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:37.513227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:37.606690 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-532582bf-3406-4fa8-8db1-020102419716 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4d6545231d145a297bb3aec14275033', 'request_id': 'req-532582bf-3406-4fa8-8db1-020102419716'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:37.613856 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-532582bf-3406-4fa8-8db1-020102419716 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4d6545231d145a297bb3aec14275033', 'request_id': 'req-532582bf-3406-4fa8-8db1-020102419716'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:37.620048 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-532582bf-3406-4fa8-8db1-020102419716 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4d6545231d145a297bb3aec14275033', 'request_id': 'req-532582bf-3406-4fa8-8db1-020102419716'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:37.703142 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-532582bf-3406-4fa8-8db1-020102419716 admin admin] The token cache is being invalidate because project e4d6545231d145a297bb3aec14275033 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:37.703463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-532582bf-3406-4fa8-8db1-020102419716 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:37.704118 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-532582bf-3406-4fa8-8db1-020102419716 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:37.705659 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1263/2525] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:37 2022] DELETE /identity/v3/projects/e4d6545231d145a297bb3aec14275033 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:37.709684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac465e2-fbb8-420b-9299-4aac829c3802 service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:37.720398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac465e2-fbb8-420b-9299-4aac829c3802 service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:37.748087 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:37.775549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c712f-629f-4d7e-8caf-45876dcd1340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:37.776392 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c712f-629f-4d7e-8caf-45876dcd1340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:37.776671 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c712f-629f-4d7e-8caf-45876dcd1340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:37.776942 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c712f-629f-4d7e-8caf-45876dcd1340 admin admin] PATH_INFO: `/v3/users/f6744a895492446a8523a634dd174c88` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:37.778113 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c712f-629f-4d7e-8caf-45876dcd1340 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6744a895492446a8523a634dd174c88)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:37.780045 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c712f-629f-4d7e-8caf-45876dcd1340 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:37.930706 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-ca8c712f-629f-4d7e-8caf-45876dcd1340 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6744a895492446a8523a634dd174c88'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:37.979691 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-ca8c712f-629f-4d7e-8caf-45876dcd1340 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6744a895492446a8523a634dd174c88'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:37.988118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-ca8c712f-629f-4d7e-8caf-45876dcd1340 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6744a895492446a8523a634dd174c88'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:38.001612 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-ca8c712f-629f-4d7e-8caf-45876dcd1340 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6744a895492446a8523a634dd174c88'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:38.002330 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-ca8c712f-629f-4d7e-8caf-45876dcd1340 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6744a895492446a8523a634dd174c88'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:38.009454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1263/2526] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:37 2022] DELETE /identity/v3/users/f6744a895492446a8523a634dd174c88 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:38.470811 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532582bf-3406-4fa8-8db1-020102419716 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:38.472419 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-532582bf-3406-4fa8-8db1-020102419716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:38.486056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532582bf-3406-4fa8-8db1-020102419716 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:38.557013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abcf96e-9854-4fd6-90fc-4a9acca3643c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:38.557013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcf96e-9854-4fd6-90fc-4a9acca3643c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:38.557013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcf96e-9854-4fd6-90fc-4a9acca3643c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:38.557013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcf96e-9854-4fd6-90fc-4a9acca3643c admin admin] PATH_INFO: `/v3/projects/062c7dd4a5784c4abb5b12ec72ee75aa` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:38.557829 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcf96e-9854-4fd6-90fc-4a9acca3643c admin admin] RBAC: Authorizing `identity:delete_project(project_id=062c7dd4a5784c4abb5b12ec72ee75aa)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:38.559813 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcf96e-9854-4fd6-90fc-4a9acca3643c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:38.671024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2abcf96e-9854-4fd6-90fc-4a9acca3643c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '062c7dd4a5784c4abb5b12ec72ee75aa', 'request_id': 'req-2abcf96e-9854-4fd6-90fc-4a9acca3643c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:38.677076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2abcf96e-9854-4fd6-90fc-4a9acca3643c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '062c7dd4a5784c4abb5b12ec72ee75aa', 'request_id': 'req-2abcf96e-9854-4fd6-90fc-4a9acca3643c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:38.683137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2abcf96e-9854-4fd6-90fc-4a9acca3643c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '062c7dd4a5784c4abb5b12ec72ee75aa', 'request_id': 'req-2abcf96e-9854-4fd6-90fc-4a9acca3643c'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:38.767757 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2abcf96e-9854-4fd6-90fc-4a9acca3643c admin admin] The token cache is being invalidate because project 062c7dd4a5784c4abb5b12ec72ee75aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:38.768042 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2abcf96e-9854-4fd6-90fc-4a9acca3643c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:38.768658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-2abcf96e-9854-4fd6-90fc-4a9acca3643c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:38.770199 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1264/2527] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:38 2022] DELETE /identity/v3/projects/062c7dd4a5784c4abb5b12ec72ee75aa => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:38.775131 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a9c58-6f5f-42b1-886f-2e2eb2408696 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:38.775598 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a9c58-6f5f-42b1-886f-2e2eb2408696 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:38.793452 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a9c58-6f5f-42b1-886f-2e2eb2408696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:38.794875 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-653a9c58-6f5f-42b1-886f-2e2eb2408696 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:38.796104 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-653a9c58-6f5f-42b1-886f-2e2eb2408696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:38.855807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1264/2528] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:38.864061 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a3ef0-f95b-4218-80d7-e66aa79b2d19 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:38.864421 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a3ef0-f95b-4218-80d7-e66aa79b2d19 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:38.864705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a3ef0-f95b-4218-80d7-e66aa79b2d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:38.885083 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f38a3ef0-f95b-4218-80d7-e66aa79b2d19 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:38.886330 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f38a3ef0-f95b-4218-80d7-e66aa79b2d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:38.895422 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1265/2529] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:38.899891 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a9c58-6f5f-42b1-886f-2e2eb2408696 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:38.911109 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a9c58-6f5f-42b1-886f-2e2eb2408696 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:38.915744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b1540-d472-4fbc-afed-d6d0174a0569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:38.916572 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b1540-d472-4fbc-afed-d6d0174a0569 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:38.916852 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b1540-d472-4fbc-afed-d6d0174a0569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:38.917137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b1540-d472-4fbc-afed-d6d0174a0569 admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:38.917874 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b1540-d472-4fbc-afed-d6d0174a0569 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:48:38.918181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b1540-d472-4fbc-afed-d6d0174a0569 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:38.919822 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b1540-d472-4fbc-afed-d6d0174a0569 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:38.932656 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1265/2530] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:48:38.937046 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a3ef0-f95b-4218-80d7-e66aa79b2d19 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:38.946444 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a3ef0-f95b-4218-80d7-e66aa79b2d19 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:38.951143 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1b70c-c8b7-4898-9074-65d71276e616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:38.951796 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1b70c-c8b7-4898-9074-65d71276e616 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:38.952081 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1b70c-c8b7-4898-9074-65d71276e616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:38.952354 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1b70c-c8b7-4898-9074-65d71276e616 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:38.953066 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1b70c-c8b7-4898-9074-65d71276e616 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:38.955014 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1b70c-c8b7-4898-9074-65d71276e616 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:39.061481 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1266/2531] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:38 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:39.062755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b1540-d472-4fbc-afed-d6d0174a0569 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:39.083847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b1540-d472-4fbc-afed-d6d0174a0569 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:39.124667 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:39.153051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10647f08-f2a2-430d-bf65-0764bb986206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:39.153868 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10647f08-f2a2-430d-bf65-0764bb986206 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:39.154144 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10647f08-f2a2-430d-bf65-0764bb986206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:39.154421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10647f08-f2a2-430d-bf65-0764bb986206 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:39.155419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10647f08-f2a2-430d-bf65-0764bb986206 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:39.159357 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10647f08-f2a2-430d-bf65-0764bb986206 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:39.252962 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1266/2532] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:39 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:39.258014 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1b70c-c8b7-4898-9074-65d71276e616 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:39.265903 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:39.273261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1b70c-c8b7-4898-9074-65d71276e616 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:39.279386 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd996b65-7ab6-4f86-b0fe-dd8e44187299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:39.280063 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd996b65-7ab6-4f86-b0fe-dd8e44187299 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:39.280482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd996b65-7ab6-4f86-b0fe-dd8e44187299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:39.281052 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd996b65-7ab6-4f86-b0fe-dd8e44187299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:39.281818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd996b65-7ab6-4f86-b0fe-dd8e44187299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:39.284049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd996b65-7ab6-4f86-b0fe-dd8e44187299 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:39.296786 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1267/2533] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:39.305790 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10647f08-f2a2-430d-bf65-0764bb986206 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:39.317205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10647f08-f2a2-430d-bf65-0764bb986206 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:39.322409 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bcb06-84b9-4a36-ac18-17a112050260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:39.323206 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bcb06-84b9-4a36-ac18-17a112050260 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:39.323487 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bcb06-84b9-4a36-ac18-17a112050260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:39.323778 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bcb06-84b9-4a36-ac18-17a112050260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:39.324518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571bcb06-84b9-4a36-ac18-17a112050260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:39.326244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571bcb06-84b9-4a36-ac18-17a112050260 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:39.341645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1267/2534] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:39.346637 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd996b65-7ab6-4f86-b0fe-dd8e44187299 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:39.359156 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd996b65-7ab6-4f86-b0fe-dd8e44187299 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:39.365950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b1f04-28b8-4b9f-92d1-40793e7f993e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:39.367443 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b1f04-28b8-4b9f-92d1-40793e7f993e admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:39.367443 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b1f04-28b8-4b9f-92d1-40793e7f993e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:39.367683 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b1f04-28b8-4b9f-92d1-40793e7f993e admin admin] PATH_INFO: `/v3/projects/a65fcbc24b0a4f3aba69b909298f267c/users/7ffc5e8c234e469d944d9a2cc2795581/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:39.384897 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b1f04-28b8-4b9f-92d1-40793e7f993e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a65fcbc24b0a4f3aba69b909298f267c, user_id=7ffc5e8c234e469d944d9a2cc2795581, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:39.388391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b1f04-28b8-4b9f-92d1-40793e7f993e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:39.490445 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1268/2535] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:39 2022] PUT /identity/v3/projects/a65fcbc24b0a4f3aba69b909298f267c/users/7ffc5e8c234e469d944d9a2cc2795581/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:39.569243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb923f-c709-4355-b8c2-9cfd17d027be None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:39.569536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb923f-c709-4355-b8c2-9cfd17d027be None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:39.569818 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb923f-c709-4355-b8c2-9cfd17d027be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:39.598967 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-4ddb923f-c709-4355-b8c2-9cfd17d027be None None] MFA Rules not processed for user `7ffc5e8c234e469d944d9a2cc2795581`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:39.600158 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-4ddb923f-c709-4355-b8c2-9cfd17d027be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:39.674834 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1268/2536] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:39.679796 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b1f04-28b8-4b9f-92d1-40793e7f993e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:39.753828 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b1f04-28b8-4b9f-92d1-40793e7f993e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:39.758541 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eec680-ef65-4add-87e0-99338955744d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:39.759237 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eec680-ef65-4add-87e0-99338955744d admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:39.759521 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eec680-ef65-4add-87e0-99338955744d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:39.759796 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eec680-ef65-4add-87e0-99338955744d admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:39.760503 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eec680-ef65-4add-87e0-99338955744d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:39.762412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eec680-ef65-4add-87e0-99338955744d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:39.860500 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1269/2537] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:39 2022] POST /identity/v3/projects => generated 385 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:39.865148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb923f-c709-4355-b8c2-9cfd17d027be None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:39.876907 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb923f-c709-4355-b8c2-9cfd17d027be None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:39.943125 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42b53b-ad6d-46dc-9639-64812d83a19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:39.943804 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42b53b-ad6d-46dc-9639-64812d83a19c admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:39.944086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42b53b-ad6d-46dc-9639-64812d83a19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:39.944504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42b53b-ad6d-46dc-9639-64812d83a19c admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:39.946363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f42b53b-ad6d-46dc-9639-64812d83a19c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:39.946997 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f42b53b-ad6d-46dc-9639-64812d83a19c admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:40.058697 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1269/2538] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:39 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:40.062847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eec680-ef65-4add-87e0-99338955744d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:40.073766 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eec680-ef65-4add-87e0-99338955744d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:40.078483 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c923909-007e-4fb2-b740-2aa1ae9e062a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:40.079319 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c923909-007e-4fb2-b740-2aa1ae9e062a admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:40.079621 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c923909-007e-4fb2-b740-2aa1ae9e062a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:40.079902 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c923909-007e-4fb2-b740-2aa1ae9e062a admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:40.080679 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c923909-007e-4fb2-b740-2aa1ae9e062a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:40.082278 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c923909-007e-4fb2-b740-2aa1ae9e062a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:40.093173 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1270/2539] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:40.097724 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42b53b-ad6d-46dc-9639-64812d83a19c admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:40.107783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42b53b-ad6d-46dc-9639-64812d83a19c admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:40.113350 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc74a8-0548-4a91-8828-5dae39ec955a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:40.113948 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc74a8-0548-4a91-8828-5dae39ec955a admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:40.114258 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc74a8-0548-4a91-8828-5dae39ec955a admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:40.114541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc74a8-0548-4a91-8828-5dae39ec955a admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:40.115307 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc74a8-0548-4a91-8828-5dae39ec955a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:40.116912 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc74a8-0548-4a91-8828-5dae39ec955a admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:40.127213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1270/2540] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:40.131513 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c923909-007e-4fb2-b740-2aa1ae9e062a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:40.141473 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c923909-007e-4fb2-b740-2aa1ae9e062a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:40.146616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a27119-ead1-4e86-9c6f-3df63253710b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:40.147605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a27119-ead1-4e86-9c6f-3df63253710b admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:40.147884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a27119-ead1-4e86-9c6f-3df63253710b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:40.148153 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a27119-ead1-4e86-9c6f-3df63253710b admin admin] PATH_INFO: `/v3/projects/fd6d3810bcc14bb694f8f9b43325bd7e/users/a2d8309114ef416595b218dcc37331d4/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:40.167163 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a27119-ead1-4e86-9c6f-3df63253710b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd6d3810bcc14bb694f8f9b43325bd7e, user_id=a2d8309114ef416595b218dcc37331d4, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:40.169815 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a27119-ead1-4e86-9c6f-3df63253710b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:40.237229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1271/2541] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:40 2022] PUT /identity/v3/projects/fd6d3810bcc14bb694f8f9b43325bd7e/users/a2d8309114ef416595b218dcc37331d4/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:40.248191 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75816257-1d62-49a0-9f3a-f3566048fd09 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:40.248480 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75816257-1d62-49a0-9f3a-f3566048fd09 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:40.248759 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75816257-1d62-49a0-9f3a-f3566048fd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:40.276200 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-75816257-1d62-49a0-9f3a-f3566048fd09 None None] MFA Rules not processed for user `a2d8309114ef416595b218dcc37331d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:40.277445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-75816257-1d62-49a0-9f3a-f3566048fd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:40.290712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:40.371901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1271/2542] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:48:40.377020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a27119-ead1-4e86-9c6f-3df63253710b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:40.440616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a27119-ead1-4e86-9c6f-3df63253710b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:40.445601 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d70f15-6aa1-42be-b1fd-34c933a6a0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:40.446270 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d70f15-6aa1-42be-b1fd-34c933a6a0b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:40.446563 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d70f15-6aa1-42be-b1fd-34c933a6a0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:40.446840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d70f15-6aa1-42be-b1fd-34c933a6a0b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:40.447615 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d70f15-6aa1-42be-b1fd-34c933a6a0b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:40.449545 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d70f15-6aa1-42be-b1fd-34c933a6a0b5 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:40.538937 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1272/2543] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:40 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:48:40.540692 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75816257-1d62-49a0-9f3a-f3566048fd09 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:40.553271 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75816257-1d62-49a0-9f3a-f3566048fd09 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:40.613145 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46465ef-e00a-43a2-8ca8-4681331d08cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:40.613842 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46465ef-e00a-43a2-8ca8-4681331d08cc admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:40.614121 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46465ef-e00a-43a2-8ca8-4681331d08cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:40.614395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46465ef-e00a-43a2-8ca8-4681331d08cc admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:40.615675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46465ef-e00a-43a2-8ca8-4681331d08cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:40.617733 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46465ef-e00a-43a2-8ca8-4681331d08cc admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:40.698113 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1272/2544] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:40 2022] POST /identity/v3/users => generated 299 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:40.702005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d70f15-6aa1-42be-b1fd-34c933a6a0b5 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:40.707654 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:40.712864 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d70f15-6aa1-42be-b1fd-34c933a6a0b5 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:40.717823 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882ec78-50ae-45b1-9bb2-c3ee308be935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:40.718493 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882ec78-50ae-45b1-9bb2-c3ee308be935 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:40.718777 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882ec78-50ae-45b1-9bb2-c3ee308be935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:40.719087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b882ec78-50ae-45b1-9bb2-c3ee308be935 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:40.720468 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882ec78-50ae-45b1-9bb2-c3ee308be935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:40.722193 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b882ec78-50ae-45b1-9bb2-c3ee308be935 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:40.733271 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1273/2545] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:40.738262 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46465ef-e00a-43a2-8ca8-4681331d08cc admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:40.748533 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46465ef-e00a-43a2-8ca8-4681331d08cc admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:40.753559 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51569813-18db-4a95-8a77-9d9844f2d035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:40.754501 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51569813-18db-4a95-8a77-9d9844f2d035 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:40.754782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51569813-18db-4a95-8a77-9d9844f2d035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:40.755117 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51569813-18db-4a95-8a77-9d9844f2d035 admin admin] PATH_INFO: `/v3/projects/78a94d4412b546c9abc56a6ea233a4d9/users/22dcb67ff7eb443fb8fd6f036d3d2e7a/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:40.771139 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51569813-18db-4a95-8a77-9d9844f2d035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78a94d4412b546c9abc56a6ea233a4d9, user_id=22dcb67ff7eb443fb8fd6f036d3d2e7a, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:40.777336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51569813-18db-4a95-8a77-9d9844f2d035 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:40.854531 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1273/2546] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:40 2022] PUT /identity/v3/projects/78a94d4412b546c9abc56a6ea233a4d9/users/22dcb67ff7eb443fb8fd6f036d3d2e7a/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:40.858689 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882ec78-50ae-45b1-9bb2-c3ee308be935 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:40.869427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b882ec78-50ae-45b1-9bb2-c3ee308be935 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:40.936962 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847c297-a446-4a05-b09f-f02135f78dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:40.937665 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847c297-a446-4a05-b09f-f02135f78dea admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:40.938034 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847c297-a446-4a05-b09f-f02135f78dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:40.938647 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847c297-a446-4a05-b09f-f02135f78dea admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:40.939150 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847c297-a446-4a05-b09f-f02135f78dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:40.940744 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847c297-a446-4a05-b09f-f02135f78dea admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:40.954652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1274/2547] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:48:40.957714 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51569813-18db-4a95-8a77-9d9844f2d035 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:40.968405 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51569813-18db-4a95-8a77-9d9844f2d035 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:40.974092 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e5663-166a-43e7-8d8f-72ee711a5183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:40.975090 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e5663-166a-43e7-8d8f-72ee711a5183 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:40.975321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e5663-166a-43e7-8d8f-72ee711a5183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:40.975593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e5663-166a-43e7-8d8f-72ee711a5183 admin admin] PATH_INFO: `/v3/projects/78a94d4412b546c9abc56a6ea233a4d9/users/22dcb67ff7eb443fb8fd6f036d3d2e7a/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:40.977708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e5663-166a-43e7-8d8f-72ee711a5183 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78a94d4412b546c9abc56a6ea233a4d9, user_id=22dcb67ff7eb443fb8fd6f036d3d2e7a, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:40.980588 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e5663-166a-43e7-8d8f-72ee711a5183 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:40.993854 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1274/2548] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:40 2022] PUT /identity/v3/projects/78a94d4412b546c9abc56a6ea233a4d9/users/22dcb67ff7eb443fb8fd6f036d3d2e7a/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:41.005982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a8e7d-791b-4a51-a157-f751a17b7843 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:41.006301 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a8e7d-791b-4a51-a157-f751a17b7843 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:41.006769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a8e7d-791b-4a51-a157-f751a17b7843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:41.034131 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-d70a8e7d-791b-4a51-a157-f751a17b7843 None None] MFA Rules not processed for user `22dcb67ff7eb443fb8fd6f036d3d2e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:41.035451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-d70a8e7d-791b-4a51-a157-f751a17b7843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:41.104267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1275/2549] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:41.115565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e5663-166a-43e7-8d8f-72ee711a5183 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:41.117272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-d45e5663-166a-43e7-8d8f-72ee711a5183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:41.175347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e5663-166a-43e7-8d8f-72ee711a5183 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:41.179248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09943c03-7f28-4e1a-b443-e7f582dfa30b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:41.179892 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09943c03-7f28-4e1a-b443-e7f582dfa30b service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:41.180171 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09943c03-7f28-4e1a-b443-e7f582dfa30b service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:41.180443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09943c03-7f28-4e1a-b443-e7f582dfa30b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:41.191102 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09943c03-7f28-4e1a-b443-e7f582dfa30b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:41.192919 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09943c03-7f28-4e1a-b443-e7f582dfa30b service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:41.204609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1275/2550] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:41 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:48:42.265742 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a8e7d-791b-4a51-a157-f751a17b7843 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:42.275752 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a8e7d-791b-4a51-a157-f751a17b7843 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:42.279398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d3f068-3d5d-4096-be84-508e4560b541 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:42.280058 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3f068-3d5d-4096-be84-508e4560b541 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:42.280335 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3f068-3d5d-4096-be84-508e4560b541 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:42.280861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d3f068-3d5d-4096-be84-508e4560b541 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:42.339513 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d3f068-3d5d-4096-be84-508e4560b541 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:42.340993 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d3f068-3d5d-4096-be84-508e4560b541 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:42.350990 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1276/2551] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:48:43.357737 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09943c03-7f28-4e1a-b443-e7f582dfa30b service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:43.359290 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-09943c03-7f28-4e1a-b443-e7f582dfa30b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:43.373312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:43.434399 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09943c03-7f28-4e1a-b443-e7f582dfa30b service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:43.439399 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fcdd48-0546-46fc-a695-05c0c3e8e716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:43.440303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcdd48-0546-46fc-a695-05c0c3e8e716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:43.440620 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcdd48-0546-46fc-a695-05c0c3e8e716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:43.441081 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcdd48-0546-46fc-a695-05c0c3e8e716 admin admin] PATH_INFO: `/v3/users/b6e5c6fbf5244a31817c5bdb3f2b4764` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:43.442532 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fcdd48-0546-46fc-a695-05c0c3e8e716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6e5c6fbf5244a31817c5bdb3f2b4764)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:43.444561 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fcdd48-0546-46fc-a695-05c0c3e8e716 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:43.658023 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-b7fcdd48-0546-46fc-a695-05c0c3e8e716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6e5c6fbf5244a31817c5bdb3f2b4764'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:43.750947 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-b7fcdd48-0546-46fc-a695-05c0c3e8e716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6e5c6fbf5244a31817c5bdb3f2b4764'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:43.758489 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-b7fcdd48-0546-46fc-a695-05c0c3e8e716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6e5c6fbf5244a31817c5bdb3f2b4764'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:43.771212 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-b7fcdd48-0546-46fc-a695-05c0c3e8e716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6e5c6fbf5244a31817c5bdb3f2b4764'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:43.771886 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-b7fcdd48-0546-46fc-a695-05c0c3e8e716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6e5c6fbf5244a31817c5bdb3f2b4764'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:43.777960 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1276/2552] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:43 2022] DELETE /identity/v3/users/b6e5c6fbf5244a31817c5bdb3f2b4764 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:43.783739 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eedf06-f72b-4f8b-9be1-8548f1653a8a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:43.784230 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eedf06-f72b-4f8b-9be1-8548f1653a8a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:43.784331 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eedf06-f72b-4f8b-9be1-8548f1653a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:43.801489 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b5eedf06-f72b-4f8b-9be1-8548f1653a8a None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:43.802687 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b5eedf06-f72b-4f8b-9be1-8548f1653a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:43.807698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:43.860754 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1277/2553] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:43.870705 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fcdd48-0546-46fc-a695-05c0c3e8e716 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:43.872155 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b7fcdd48-0546-46fc-a695-05c0c3e8e716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:43.929937 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fcdd48-0546-46fc-a695-05c0c3e8e716 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:43.933603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8eb0be-aedd-4801-8503-039f4e557a66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:43.934243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8eb0be-aedd-4801-8503-039f4e557a66 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:43.934524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8eb0be-aedd-4801-8503-039f4e557a66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:43.934801 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8eb0be-aedd-4801-8503-039f4e557a66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:43.943985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8eb0be-aedd-4801-8503-039f4e557a66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:43.945457 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8eb0be-aedd-4801-8503-039f4e557a66 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:43.956426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1277/2554] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:48:44.358322 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eedf06-f72b-4f8b-9be1-8548f1653a8a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:44.360587 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b5eedf06-f72b-4f8b-9be1-8548f1653a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:44.373371 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eedf06-f72b-4f8b-9be1-8548f1653a8a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:44.384005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a64d7-f7cc-4fff-9353-21251165be2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:44.384005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a64d7-f7cc-4fff-9353-21251165be2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:44.384005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a64d7-f7cc-4fff-9353-21251165be2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:44.384005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a64d7-f7cc-4fff-9353-21251165be2e admin admin] PATH_INFO: `/v3/projects/d637aa134f4d4e49861ba4f8649ca8d8` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:44.385606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a64d7-f7cc-4fff-9353-21251165be2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d637aa134f4d4e49861ba4f8649ca8d8)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:44.387648 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a64d7-f7cc-4fff-9353-21251165be2e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:44.509036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a51a64d7-f7cc-4fff-9353-21251165be2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd637aa134f4d4e49861ba4f8649ca8d8', 'request_id': 'req-a51a64d7-f7cc-4fff-9353-21251165be2e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:44.515530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a51a64d7-f7cc-4fff-9353-21251165be2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd637aa134f4d4e49861ba4f8649ca8d8', 'request_id': 'req-a51a64d7-f7cc-4fff-9353-21251165be2e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:44.522432 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a51a64d7-f7cc-4fff-9353-21251165be2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd637aa134f4d4e49861ba4f8649ca8d8', 'request_id': 'req-a51a64d7-f7cc-4fff-9353-21251165be2e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:44.596664 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a51a64d7-f7cc-4fff-9353-21251165be2e admin admin] The token cache is being invalidate because project d637aa134f4d4e49861ba4f8649ca8d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:44.596951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a51a64d7-f7cc-4fff-9353-21251165be2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:44.597567 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-a51a64d7-f7cc-4fff-9353-21251165be2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:44.599135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1278/2555] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:44 2022] DELETE /identity/v3/projects/d637aa134f4d4e49861ba4f8649ca8d8 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:44.603616 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8eb0be-aedd-4801-8503-039f4e557a66 service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:44.613426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8eb0be-aedd-4801-8503-039f4e557a66 service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:44.642513 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:44.671050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c2d87-8ebc-4746-a900-8aba0585aaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:44.671855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c2d87-8ebc-4746-a900-8aba0585aaaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:44.672135 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c2d87-8ebc-4746-a900-8aba0585aaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:44.672423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c2d87-8ebc-4746-a900-8aba0585aaaf admin admin] PATH_INFO: `/v3/users/550c8de5c73a4378abfa50ac411953a2` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:44.673792 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c2d87-8ebc-4746-a900-8aba0585aaaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=550c8de5c73a4378abfa50ac411953a2)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:44.675988 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c2d87-8ebc-4746-a900-8aba0585aaaf admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:44.904821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c66c2d87-8ebc-4746-a900-8aba0585aaaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '550c8de5c73a4378abfa50ac411953a2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:44.985325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c66c2d87-8ebc-4746-a900-8aba0585aaaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '550c8de5c73a4378abfa50ac411953a2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:44.993408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c66c2d87-8ebc-4746-a900-8aba0585aaaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '550c8de5c73a4378abfa50ac411953a2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:45.006756 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c66c2d87-8ebc-4746-a900-8aba0585aaaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '550c8de5c73a4378abfa50ac411953a2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:45.007594 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c66c2d87-8ebc-4746-a900-8aba0585aaaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '550c8de5c73a4378abfa50ac411953a2'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:45.008112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4609e-6cc7-488d-824d-52dfe891b14b None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:45.008587 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4609e-6cc7-488d-824d-52dfe891b14b None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:45.008678 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f4609e-6cc7-488d-824d-52dfe891b14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:45.013715 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1278/2556] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:44 2022] DELETE /identity/v3/users/550c8de5c73a4378abfa50ac411953a2 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:45.030338 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a9f4609e-6cc7-488d-824d-52dfe891b14b None None] MFA Rules not processed for user `a4f499707998403d8f833eb783599466`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:45.032010 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a9f4609e-6cc7-488d-824d-52dfe891b14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:45.090409 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1279/2557] 10.222.0.27 () {58 vars in 1009 bytes} [Sat Jul 16 15:48:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:48:45.225006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c2d87-8ebc-4746-a900-8aba0585aaaf admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:45.226411 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c66c2d87-8ebc-4746-a900-8aba0585aaaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:45.303723 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c2d87-8ebc-4746-a900-8aba0585aaaf admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:45.309147 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06401813-6ac1-4a38-ac90-b0e268838bf2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1464c83fb294a3d90e5763ef648e834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:45.309866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06401813-6ac1-4a38-ac90-b0e268838bf2 service swift] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:45.310149 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06401813-6ac1-4a38-ac90-b0e268838bf2 service swift] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:45.310423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06401813-6ac1-4a38-ac90-b0e268838bf2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:45.321575 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06401813-6ac1-4a38-ac90-b0e268838bf2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:45.323218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06401813-6ac1-4a38-ac90-b0e268838bf2 service swift] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:45.333972 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1279/2558] 10.222.0.27 () {60 vars in 1384 bytes} [Sat Jul 16 15:48:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 15:48:45.701385 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f4609e-6cc7-488d-824d-52dfe891b14b None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:45.703208 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a9f4609e-6cc7-488d-824d-52dfe891b14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:45.751977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:45.775014 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f4609e-6cc7-488d-824d-52dfe891b14b None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:45.779545 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55ec56-77c9-4886-884b-4efd7c57ae74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:45.785031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55ec56-77c9-4886-884b-4efd7c57ae74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:45.785031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55ec56-77c9-4886-884b-4efd7c57ae74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:45.785031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55ec56-77c9-4886-884b-4efd7c57ae74 admin admin] PATH_INFO: `/v3/projects/d34ffb4f71c142b991375e02a7030694` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:45.785031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c55ec56-77c9-4886-884b-4efd7c57ae74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d34ffb4f71c142b991375e02a7030694)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:45.785031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c55ec56-77c9-4886-884b-4efd7c57ae74 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:45.942328 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5c55ec56-77c9-4886-884b-4efd7c57ae74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd34ffb4f71c142b991375e02a7030694', 'request_id': 'req-5c55ec56-77c9-4886-884b-4efd7c57ae74'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:45.947991 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5c55ec56-77c9-4886-884b-4efd7c57ae74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd34ffb4f71c142b991375e02a7030694', 'request_id': 'req-5c55ec56-77c9-4886-884b-4efd7c57ae74'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:45.954240 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5c55ec56-77c9-4886-884b-4efd7c57ae74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd34ffb4f71c142b991375e02a7030694', 'request_id': 'req-5c55ec56-77c9-4886-884b-4efd7c57ae74'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:46.028289 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5c55ec56-77c9-4886-884b-4efd7c57ae74 admin admin] The token cache is being invalidate because project d34ffb4f71c142b991375e02a7030694 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:46.028289 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5c55ec56-77c9-4886-884b-4efd7c57ae74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:46.029811 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5c55ec56-77c9-4886-884b-4efd7c57ae74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:46.031914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1280/2559] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:45 2022] DELETE /identity/v3/projects/d34ffb4f71c142b991375e02a7030694 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:46.037441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef76bdb-dd9a-4d3a-a2dc-81d115d0e4a0 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:46.038345 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef76bdb-dd9a-4d3a-a2dc-81d115d0e4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:46.038629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef76bdb-dd9a-4d3a-a2dc-81d115d0e4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:46.058777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-3ef76bdb-dd9a-4d3a-a2dc-81d115d0e4a0 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:46.059899 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3ef76bdb-dd9a-4d3a-a2dc-81d115d0e4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:46.095918 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:46.127014 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1280/2560] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:46.135200 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a29eb-2026-456f-97b4-641b3fa01387 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:46.135727 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a29eb-2026-456f-97b4-641b3fa01387 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:46.136008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a29eb-2026-456f-97b4-641b3fa01387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:46.155113 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-749a29eb-2026-456f-97b4-641b3fa01387 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:46.156577 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-749a29eb-2026-456f-97b4-641b3fa01387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:46.165182 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1281/2561] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:46.176968 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef76bdb-dd9a-4d3a-a2dc-81d115d0e4a0 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:46.186639 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef76bdb-dd9a-4d3a-a2dc-81d115d0e4a0 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:46.192083 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e7474-67fc-4140-84d1-ad1f97448891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:46.192771 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e7474-67fc-4140-84d1-ad1f97448891 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:46.193153 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e7474-67fc-4140-84d1-ad1f97448891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:46.193440 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e7474-67fc-4140-84d1-ad1f97448891 admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:46.194166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e7474-67fc-4140-84d1-ad1f97448891 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:48:46.194471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e7474-67fc-4140-84d1-ad1f97448891 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:46.196106 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e7474-67fc-4140-84d1-ad1f97448891 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:46.209253 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1281/2562] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 15:48:46.213715 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a29eb-2026-456f-97b4-641b3fa01387 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:46.223512 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a29eb-2026-456f-97b4-641b3fa01387 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:46.228123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31457f1d-dec2-4e99-b4fc-bad224eb7de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:46.228760 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31457f1d-dec2-4e99-b4fc-bad224eb7de0 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:46.229043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31457f1d-dec2-4e99-b4fc-bad224eb7de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:46.229316 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31457f1d-dec2-4e99-b4fc-bad224eb7de0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:46.230053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31457f1d-dec2-4e99-b4fc-bad224eb7de0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:46.231994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31457f1d-dec2-4e99-b4fc-bad224eb7de0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:46.242915 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f634e-458e-49f8-986f-383a9dfebee9 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:46.243205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f634e-458e-49f8-986f-383a9dfebee9 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:46.243486 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f634e-458e-49f8-986f-383a9dfebee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:46.265415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-d78f634e-458e-49f8-986f-383a9dfebee9 None None] MFA Rules not processed for user `a4f499707998403d8f833eb783599466`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:46.266633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-d78f634e-458e-49f8-986f-383a9dfebee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:46.326274 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1282/2563] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:46 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:48:46.330365 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31457f1d-dec2-4e99-b4fc-bad224eb7de0 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:46.345243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1282/2564] 10.222.0.27 () {58 vars in 967 bytes} [Sat Jul 16 15:48:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:48:46.363252 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f634e-458e-49f8-986f-383a9dfebee9 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:46.365035 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-d78f634e-458e-49f8-986f-383a9dfebee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:46.402997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31457f1d-dec2-4e99-b4fc-bad224eb7de0 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:46.409242 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6c9fb-6b36-41f3-bafe-4e0770689062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:46.411053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6c9fb-6b36-41f3-bafe-4e0770689062 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:46.411233 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6c9fb-6b36-41f3-bafe-4e0770689062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:46.411495 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6c9fb-6b36-41f3-bafe-4e0770689062 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:46.412251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6c9fb-6b36-41f3-bafe-4e0770689062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:46.414435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6c9fb-6b36-41f3-bafe-4e0770689062 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:46.443116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f634e-458e-49f8-986f-383a9dfebee9 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:46.511663 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fa023-8bc4-42e5-a369-6b66fc7847e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1464c83fb294a3d90e5763ef648e834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:46.512292 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fa023-8bc4-42e5-a369-6b66fc7847e5 service swift] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:46.512623 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fa023-8bc4-42e5-a369-6b66fc7847e5 service swift] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:46.512896 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fa023-8bc4-42e5-a369-6b66fc7847e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:46.538220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:46.572624 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27fa023-8bc4-42e5-a369-6b66fc7847e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:46.574217 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27fa023-8bc4-42e5-a369-6b66fc7847e5 service swift] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:46.583046 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1283/2565] 10.222.0.27 () {60 vars in 1384 bytes} [Sat Jul 16 15:48:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 16 15:48:46.745211 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1283/2566] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:46 2022] POST /identity/v3/users => generated 293 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:46.749411 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fa023-8bc4-42e5-a369-6b66fc7847e5 service swift] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:46.760419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fa023-8bc4-42e5-a369-6b66fc7847e5 service swift] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:46.767904 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311fe93-2e46-4c9b-a84d-33c942217d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:46.767904 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311fe93-2e46-4c9b-a84d-33c942217d7c admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:46.767904 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311fe93-2e46-4c9b-a84d-33c942217d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:46.767904 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311fe93-2e46-4c9b-a84d-33c942217d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:46.769234 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311fe93-2e46-4c9b-a84d-33c942217d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:46.769638 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311fe93-2e46-4c9b-a84d-33c942217d7c admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:46.779987 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1284/2567] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:46.784265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6c9fb-6b36-41f3-bafe-4e0770689062 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:46.794713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6c9fb-6b36-41f3-bafe-4e0770689062 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:46.799409 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5abc7-e603-4045-a56d-c85f7f6e4388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:46.800170 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5abc7-e603-4045-a56d-c85f7f6e4388 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:46.800477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5abc7-e603-4045-a56d-c85f7f6e4388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:46.800753 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5abc7-e603-4045-a56d-c85f7f6e4388 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:46.801583 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5abc7-e603-4045-a56d-c85f7f6e4388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:46.803269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5abc7-e603-4045-a56d-c85f7f6e4388 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:46.813205 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1284/2568] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:46.818426 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311fe93-2e46-4c9b-a84d-33c942217d7c admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:46.828615 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311fe93-2e46-4c9b-a84d-33c942217d7c admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:46.834024 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5029a-c967-4266-b4a2-635e5495d9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:46.835959 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5029a-c967-4266-b4a2-635e5495d9ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:46.835959 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5029a-c967-4266-b4a2-635e5495d9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:46.835959 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5029a-c967-4266-b4a2-635e5495d9ca admin admin] PATH_INFO: `/v3/projects/61ff93b9bf1546c7bb15f30d52db8c5b/users/04912f3232a54347aedc10d10303041a/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:46.850910 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5029a-c967-4266-b4a2-635e5495d9ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=61ff93b9bf1546c7bb15f30d52db8c5b, user_id=04912f3232a54347aedc10d10303041a, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:46.853473 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5029a-c967-4266-b4a2-635e5495d9ca admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:46.917043 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1285/2569] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:46 2022] PUT /identity/v3/projects/61ff93b9bf1546c7bb15f30d52db8c5b/users/04912f3232a54347aedc10d10303041a/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:46.928410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80046ee6-af24-4941-ac08-38f2c29d2ed9 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:46.928724 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80046ee6-af24-4941-ac08-38f2c29d2ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:46.929019 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80046ee6-af24-4941-ac08-38f2c29d2ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:46.955779 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-80046ee6-af24-4941-ac08-38f2c29d2ed9 None None] MFA Rules not processed for user `04912f3232a54347aedc10d10303041a`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:46.957634 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-80046ee6-af24-4941-ac08-38f2c29d2ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:46.966699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:47.027514 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1285/2570] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:47.032363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5029a-c967-4266-b4a2-635e5495d9ca admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:47.100333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5029a-c967-4266-b4a2-635e5495d9ca admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:47.105173 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740b1ef-2791-4d5a-93b7-af8ca4c7412f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:47.105841 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740b1ef-2791-4d5a-93b7-af8ca4c7412f admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:47.106128 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740b1ef-2791-4d5a-93b7-af8ca4c7412f admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:47.106403 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740b1ef-2791-4d5a-93b7-af8ca4c7412f admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:47.107137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740b1ef-2791-4d5a-93b7-af8ca4c7412f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:47.109044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740b1ef-2791-4d5a-93b7-af8ca4c7412f admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:47.204492 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1286/2571] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:47 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:47.209420 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80046ee6-af24-4941-ac08-38f2c29d2ed9 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:47.220331 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80046ee6-af24-4941-ac08-38f2c29d2ed9 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:47.276918 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6f258-7563-476d-8aca-eb62977f2409 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:47.277489 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6f258-7563-476d-8aca-eb62977f2409 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:47.277873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6f258-7563-476d-8aca-eb62977f2409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:47.285765 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f08c8b-079c-45a1-a1ca-e9c7b90a4761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:47.286458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f08c8b-079c-45a1-a1ca-e9c7b90a4761 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:47.286786 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f08c8b-079c-45a1-a1ca-e9c7b90a4761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:47.287127 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f08c8b-079c-45a1-a1ca-e9c7b90a4761 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:47.287799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f08c8b-079c-45a1-a1ca-e9c7b90a4761 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:47.289744 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f08c8b-079c-45a1-a1ca-e9c7b90a4761 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:47.300688 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-06d6f258-7563-476d-8aca-eb62977f2409 None None] MFA Rules not processed for user `a4f499707998403d8f833eb783599466`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:47.310760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-06d6f258-7563-476d-8aca-eb62977f2409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:47.357818 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1287/2572] 10.222.0.27 () {58 vars in 1009 bytes} [Sat Jul 16 15:48:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:48:47.370504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6f258-7563-476d-8aca-eb62977f2409 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:47.383109 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:47.400308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1286/2573] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:47 2022] POST /identity/v3/users => generated 299 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:47.400308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f08c8b-079c-45a1-a1ca-e9c7b90a4761 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:47.401005 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e2f08c8b-079c-45a1-a1ca-e9c7b90a4761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:47.443517 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6f258-7563-476d-8aca-eb62977f2409 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:47.448585 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0656b-698a-4b83-a730-4f24d237a876 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1464c83fb294a3d90e5763ef648e834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:47.449220 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0656b-698a-4b83-a730-4f24d237a876 service swift] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:47.449500 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0656b-698a-4b83-a730-4f24d237a876 service swift] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:47.449770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0656b-698a-4b83-a730-4f24d237a876 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:47.458476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0656b-698a-4b83-a730-4f24d237a876 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:47.460908 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0656b-698a-4b83-a730-4f24d237a876 service swift] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:47.463856 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f08c8b-079c-45a1-a1ca-e9c7b90a4761 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:47.469300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1288/2574] 10.222.0.27 () {60 vars in 1384 bytes} [Sat Jul 16 15:48:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 16 15:48:47.469599 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a4547-8dc8-4702-ba59-0ddd4ff018bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:47.470344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a4547-8dc8-4702-ba59-0ddd4ff018bc service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:47.470757 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a4547-8dc8-4702-ba59-0ddd4ff018bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:47.471264 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a4547-8dc8-4702-ba59-0ddd4ff018bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:47.472956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0656b-698a-4b83-a730-4f24d237a876 service swift] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:47.474109 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3f5a4547-8dc8-4702-ba59-0ddd4ff018bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:47.484420 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0656b-698a-4b83-a730-4f24d237a876 service swift] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:47.489700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602314be-84e7-4b29-8365-8f56942f74ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:47.490357 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602314be-84e7-4b29-8365-8f56942f74ac admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:47.490639 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602314be-84e7-4b29-8365-8f56942f74ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:47.490935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602314be-84e7-4b29-8365-8f56942f74ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:47.491655 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602314be-84e7-4b29-8365-8f56942f74ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:47.493230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602314be-84e7-4b29-8365-8f56942f74ac admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:47.504287 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1289/2575] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:47.508534 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602314be-84e7-4b29-8365-8f56942f74ac admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:47.518381 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602314be-84e7-4b29-8365-8f56942f74ac admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:47.523972 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45096ef-c6b7-409a-81e2-0380e8a05b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:47.525162 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45096ef-c6b7-409a-81e2-0380e8a05b62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:47.525526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45096ef-c6b7-409a-81e2-0380e8a05b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:47.525807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45096ef-c6b7-409a-81e2-0380e8a05b62 admin admin] PATH_INFO: `/v3/projects/0b1e268456bb4e9b85e69d1ee51c9c38/users/99b5e1ecc5424d87a78d261c12c65b43/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:47.537013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:47.540934 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a4547-8dc8-4702-ba59-0ddd4ff018bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:47.542383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a4547-8dc8-4702-ba59-0ddd4ff018bc service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:47.543075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45096ef-c6b7-409a-81e2-0380e8a05b62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b1e268456bb4e9b85e69d1ee51c9c38, user_id=99b5e1ecc5424d87a78d261c12c65b43, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:47.545732 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45096ef-c6b7-409a-81e2-0380e8a05b62 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:47.552499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1287/2576] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:48:47.660800 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1290/2577] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:47 2022] PUT /identity/v3/projects/0b1e268456bb4e9b85e69d1ee51c9c38/users/99b5e1ecc5424d87a78d261c12c65b43/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:47.664871 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a4547-8dc8-4702-ba59-0ddd4ff018bc service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:47.676951 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a4547-8dc8-4702-ba59-0ddd4ff018bc service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:47.735360 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05590d9-a633-41af-b538-bb577ffc81a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:47.736031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05590d9-a633-41af-b538-bb577ffc81a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:47.736312 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05590d9-a633-41af-b538-bb577ffc81a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:47.736587 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05590d9-a633-41af-b538-bb577ffc81a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:47.738074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05590d9-a633-41af-b538-bb577ffc81a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:47.739751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05590d9-a633-41af-b538-bb577ffc81a4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:47.749678 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1288/2578] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:47.754112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45096ef-c6b7-409a-81e2-0380e8a05b62 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:47.764855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45096ef-c6b7-409a-81e2-0380e8a05b62 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:47.770248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf737b8-21a5-4e9b-b728-f0966f01f93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:47.771144 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf737b8-21a5-4e9b-b728-f0966f01f93f admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:47.771421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf737b8-21a5-4e9b-b728-f0966f01f93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:47.771690 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf737b8-21a5-4e9b-b728-f0966f01f93f admin admin] PATH_INFO: `/v3/projects/0b1e268456bb4e9b85e69d1ee51c9c38/users/99b5e1ecc5424d87a78d261c12c65b43/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:47.773724 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf737b8-21a5-4e9b-b728-f0966f01f93f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b1e268456bb4e9b85e69d1ee51c9c38, user_id=99b5e1ecc5424d87a78d261c12c65b43, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:47.776387 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf737b8-21a5-4e9b-b728-f0966f01f93f admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:47.787679 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1291/2579] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:47 2022] PUT /identity/v3/projects/0b1e268456bb4e9b85e69d1ee51c9c38/users/99b5e1ecc5424d87a78d261c12c65b43/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:47.797897 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27630a-8ab3-4a84-a1b3-5e1adbb421c5 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:47.798189 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27630a-8ab3-4a84-a1b3-5e1adbb421c5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:47.798471 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27630a-8ab3-4a84-a1b3-5e1adbb421c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:47.825719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-1a27630a-8ab3-4a84-a1b3-5e1adbb421c5 None None] MFA Rules not processed for user `99b5e1ecc5424d87a78d261c12c65b43`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:47.826947 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1a27630a-8ab3-4a84-a1b3-5e1adbb421c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:47.904618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1289/2580] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:48.038107 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf737b8-21a5-4e9b-b728-f0966f01f93f admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:48.039781 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-cdf737b8-21a5-4e9b-b728-f0966f01f93f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:48.061015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba0754-df82-4d5c-aabc-0c84172d097c None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:48.061304 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba0754-df82-4d5c-aabc-0c84172d097c None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:48.061584 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba0754-df82-4d5c-aabc-0c84172d097c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:48.080768 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c3ba0754-df82-4d5c-aabc-0c84172d097c None None] MFA Rules not processed for user `a4f499707998403d8f833eb783599466`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:48.082719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c3ba0754-df82-4d5c-aabc-0c84172d097c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:48.104034 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf737b8-21a5-4e9b-b728-f0966f01f93f admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:48.107743 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3d4a3-9db8-474f-9e26-e2f34e882a22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:48.108579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3d4a3-9db8-474f-9e26-e2f34e882a22 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:48.108863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3d4a3-9db8-474f-9e26-e2f34e882a22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:48.109256 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3d4a3-9db8-474f-9e26-e2f34e882a22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:48.119020 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b3d4a3-9db8-474f-9e26-e2f34e882a22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:48.120654 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b3d4a3-9db8-474f-9e26-e2f34e882a22 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:48.131602 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1292/2581] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:48:48.141705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1290/2582] 10.222.0.27 () {58 vars in 967 bytes} [Sat Jul 16 15:48:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:48:48.157731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3d4a3-9db8-474f-9e26-e2f34e882a22 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:48.161241 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba0754-df82-4d5c-aabc-0c84172d097c None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:48.183073 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:48.216867 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:48.229370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba0754-df82-4d5c-aabc-0c84172d097c None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:48.229920 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3d4a3-9db8-474f-9e26-e2f34e882a22 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:48.234199 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc126c33-db8d-4fcb-afbb-f625c80d6e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:48.240326 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc126c33-db8d-4fcb-afbb-f625c80d6e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:48.240591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a6705-e3d9-4a09-9a6d-bb0e3e3308eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1464c83fb294a3d90e5763ef648e834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:48.240728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc126c33-db8d-4fcb-afbb-f625c80d6e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:48.240728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc126c33-db8d-4fcb-afbb-f625c80d6e04 admin admin] PATH_INFO: `/v3/users/04912f3232a54347aedc10d10303041a` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:48.240930 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a6705-e3d9-4a09-9a6d-bb0e3e3308eb service swift] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:48.240930 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a6705-e3d9-4a09-9a6d-bb0e3e3308eb service swift] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:48.240930 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a6705-e3d9-4a09-9a6d-bb0e3e3308eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:48.241241 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc126c33-db8d-4fcb-afbb-f625c80d6e04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04912f3232a54347aedc10d10303041a)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:48.241241 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc126c33-db8d-4fcb-afbb-f625c80d6e04 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:48.244770 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a6705-e3d9-4a09-9a6d-bb0e3e3308eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:48.246415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a6705-e3d9-4a09-9a6d-bb0e3e3308eb service swift] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:48.255111 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1293/2583] 10.222.0.27 () {60 vars in 1384 bytes} [Sat Jul 16 15:48:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 15:48:48.415373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fc126c33-db8d-4fcb-afbb-f625c80d6e04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04912f3232a54347aedc10d10303041a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:48.509383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fc126c33-db8d-4fcb-afbb-f625c80d6e04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04912f3232a54347aedc10d10303041a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:48.510075 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fc126c33-db8d-4fcb-afbb-f625c80d6e04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04912f3232a54347aedc10d10303041a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:48.510717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fc126c33-db8d-4fcb-afbb-f625c80d6e04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04912f3232a54347aedc10d10303041a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:48.518387 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fc126c33-db8d-4fcb-afbb-f625c80d6e04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04912f3232a54347aedc10d10303041a'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:48.536034 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1291/2584] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:48 2022] DELETE /identity/v3/users/04912f3232a54347aedc10d10303041a => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:48.540645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234a332-db81-471f-a377-4e48e2b1fe44 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:48.541010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234a332-db81-471f-a377-4e48e2b1fe44 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:48.541348 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c234a332-db81-471f-a377-4e48e2b1fe44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:48.559546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-c234a332-db81-471f-a377-4e48e2b1fe44 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:48.560877 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c234a332-db81-471f-a377-4e48e2b1fe44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:48.621835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1294/2585] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:48.633287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc126c33-db8d-4fcb-afbb-f625c80d6e04 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:48.634755 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-fc126c33-db8d-4fcb-afbb-f625c80d6e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:48.704474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc126c33-db8d-4fcb-afbb-f625c80d6e04 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:48.708969 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4300be-8be7-4cc3-85a7-fc50418e011a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:48.709708 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4300be-8be7-4cc3-85a7-fc50418e011a service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:48.709914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4300be-8be7-4cc3-85a7-fc50418e011a service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:48.710188 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4300be-8be7-4cc3-85a7-fc50418e011a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:48.719701 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4300be-8be7-4cc3-85a7-fc50418e011a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:48.721128 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4300be-8be7-4cc3-85a7-fc50418e011a service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:48.731596 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1292/2586] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:48:49.172888 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234a332-db81-471f-a377-4e48e2b1fe44 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:49.174410 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c234a332-db81-471f-a377-4e48e2b1fe44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:49.186585 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c234a332-db81-471f-a377-4e48e2b1fe44 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:49.192228 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970f162-001c-4a3c-851d-ac72bdf572d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:49.193112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970f162-001c-4a3c-851d-ac72bdf572d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:49.193395 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970f162-001c-4a3c-851d-ac72bdf572d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:49.193791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970f162-001c-4a3c-851d-ac72bdf572d0 admin admin] PATH_INFO: `/v3/projects/61ff93b9bf1546c7bb15f30d52db8c5b` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:49.198675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970f162-001c-4a3c-851d-ac72bdf572d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61ff93b9bf1546c7bb15f30d52db8c5b)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:49.198675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970f162-001c-4a3c-851d-ac72bdf572d0 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:49.355280 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c970f162-001c-4a3c-851d-ac72bdf572d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61ff93b9bf1546c7bb15f30d52db8c5b', 'request_id': 'req-c970f162-001c-4a3c-851d-ac72bdf572d0'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:49.360882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c970f162-001c-4a3c-851d-ac72bdf572d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61ff93b9bf1546c7bb15f30d52db8c5b', 'request_id': 'req-c970f162-001c-4a3c-851d-ac72bdf572d0'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:49.367914 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c970f162-001c-4a3c-851d-ac72bdf572d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61ff93b9bf1546c7bb15f30d52db8c5b', 'request_id': 'req-c970f162-001c-4a3c-851d-ac72bdf572d0'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:49.457490 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c970f162-001c-4a3c-851d-ac72bdf572d0 admin admin] The token cache is being invalidate because project 61ff93b9bf1546c7bb15f30d52db8c5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:49.457783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c970f162-001c-4a3c-851d-ac72bdf572d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:49.462217 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c970f162-001c-4a3c-851d-ac72bdf572d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:49.462217 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1295/2587] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:49 2022] DELETE /identity/v3/projects/61ff93b9bf1546c7bb15f30d52db8c5b => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:49.464918 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4300be-8be7-4cc3-85a7-fc50418e011a service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:49.478145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4300be-8be7-4cc3-85a7-fc50418e011a service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:49.535497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:49.553811 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ea63d-b4ac-4656-aec7-d2f4e79de2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:49.554723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ea63d-b4ac-4656-aec7-d2f4e79de2f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:49.555242 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ea63d-b4ac-4656-aec7-d2f4e79de2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:49.555497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ea63d-b4ac-4656-aec7-d2f4e79de2f3 admin admin] PATH_INFO: `/v3/users/99b5e1ecc5424d87a78d261c12c65b43` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:49.556696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ea63d-b4ac-4656-aec7-d2f4e79de2f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99b5e1ecc5424d87a78d261c12c65b43)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:49.558677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ea63d-b4ac-4656-aec7-d2f4e79de2f3 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:49.742037 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-144ea63d-b4ac-4656-aec7-d2f4e79de2f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99b5e1ecc5424d87a78d261c12c65b43'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:49.830315 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-144ea63d-b4ac-4656-aec7-d2f4e79de2f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99b5e1ecc5424d87a78d261c12c65b43'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:49.831933 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-144ea63d-b4ac-4656-aec7-d2f4e79de2f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99b5e1ecc5424d87a78d261c12c65b43'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:49.832659 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-144ea63d-b4ac-4656-aec7-d2f4e79de2f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99b5e1ecc5424d87a78d261c12c65b43'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:49.840666 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-144ea63d-b4ac-4656-aec7-d2f4e79de2f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99b5e1ecc5424d87a78d261c12c65b43'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:49.859307 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1293/2588] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:49 2022] DELETE /identity/v3/users/99b5e1ecc5424d87a78d261c12c65b43 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:50.264991 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970f162-001c-4a3c-851d-ac72bdf572d0 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:50.266415 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c970f162-001c-4a3c-851d-ac72bdf572d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:50.279750 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970f162-001c-4a3c-851d-ac72bdf572d0 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:50.342508 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d05692-6f2d-4dc8-b02d-4d71f35327d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:50.346920 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d05692-6f2d-4dc8-b02d-4d71f35327d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:50.346920 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d05692-6f2d-4dc8-b02d-4d71f35327d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:50.346920 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d05692-6f2d-4dc8-b02d-4d71f35327d0 admin admin] PATH_INFO: `/v3/projects/0b1e268456bb4e9b85e69d1ee51c9c38` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:50.346920 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d05692-6f2d-4dc8-b02d-4d71f35327d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b1e268456bb4e9b85e69d1ee51c9c38)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:50.348134 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d05692-6f2d-4dc8-b02d-4d71f35327d0 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:50.361437 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:50.505283 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-53d05692-6f2d-4dc8-b02d-4d71f35327d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b1e268456bb4e9b85e69d1ee51c9c38', 'request_id': 'req-53d05692-6f2d-4dc8-b02d-4d71f35327d0'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:50.510731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-53d05692-6f2d-4dc8-b02d-4d71f35327d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b1e268456bb4e9b85e69d1ee51c9c38', 'request_id': 'req-53d05692-6f2d-4dc8-b02d-4d71f35327d0'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:50.516810 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-53d05692-6f2d-4dc8-b02d-4d71f35327d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b1e268456bb4e9b85e69d1ee51c9c38', 'request_id': 'req-53d05692-6f2d-4dc8-b02d-4d71f35327d0'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:50.602722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-53d05692-6f2d-4dc8-b02d-4d71f35327d0 admin admin] The token cache is being invalidate because project 0b1e268456bb4e9b85e69d1ee51c9c38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:50.603047 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-53d05692-6f2d-4dc8-b02d-4d71f35327d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:50.603683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-53d05692-6f2d-4dc8-b02d-4d71f35327d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:50.605233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1296/2589] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:50 2022] DELETE /identity/v3/projects/0b1e268456bb4e9b85e69d1ee51c9c38 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:50.610211 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef73f5b-d6b2-4803-bfc2-fff4b3a187f5 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:50.610678 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef73f5b-d6b2-4803-bfc2-fff4b3a187f5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:50.610779 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef73f5b-d6b2-4803-bfc2-fff4b3a187f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:50.629392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-6ef73f5b-d6b2-4803-bfc2-fff4b3a187f5 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:50.630551 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6ef73f5b-d6b2-4803-bfc2-fff4b3a187f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:50.691288 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1294/2590] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:50.699144 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aef1b8-b62b-4439-a026-56933822abd4 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:50.699434 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aef1b8-b62b-4439-a026-56933822abd4 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:50.699715 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aef1b8-b62b-4439-a026-56933822abd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:50.718496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-78aef1b8-b62b-4439-a026-56933822abd4 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:50.719720 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-78aef1b8-b62b-4439-a026-56933822abd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:50.727809 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1297/2591] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:50.733760 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef73f5b-d6b2-4803-bfc2-fff4b3a187f5 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:50.742619 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef73f5b-d6b2-4803-bfc2-fff4b3a187f5 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:50.747195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950e129-a17c-495c-8ee9-4fb5a3647de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:50.747844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950e129-a17c-495c-8ee9-4fb5a3647de6 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:50.748125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950e129-a17c-495c-8ee9-4fb5a3647de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:50.748402 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950e129-a17c-495c-8ee9-4fb5a3647de6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:50.749125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950e129-a17c-495c-8ee9-4fb5a3647de6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:48:50.749452 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950e129-a17c-495c-8ee9-4fb5a3647de6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:50.751070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950e129-a17c-495c-8ee9-4fb5a3647de6 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:50.763317 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1295/2592] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:48:50.767484 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aef1b8-b62b-4439-a026-56933822abd4 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:50.776846 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aef1b8-b62b-4439-a026-56933822abd4 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:50.781820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67eb6d-f8f0-4208-b240-10469e63e044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:50.782564 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67eb6d-f8f0-4208-b240-10469e63e044 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:50.782847 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67eb6d-f8f0-4208-b240-10469e63e044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:50.783148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67eb6d-f8f0-4208-b240-10469e63e044 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:50.783842 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67eb6d-f8f0-4208-b240-10469e63e044 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:50.785761 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67eb6d-f8f0-4208-b240-10469e63e044 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:50.850187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1298/2593] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:50 2022] POST /identity/v3/projects => generated 377 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:50.854443 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950e129-a17c-495c-8ee9-4fb5a3647de6 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:50.865118 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950e129-a17c-495c-8ee9-4fb5a3647de6 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:50.912313 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:50.925031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf69bb9-8919-4d28-99c6-43af65f0cb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:50.926006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf69bb9-8919-4d28-99c6-43af65f0cb5b admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:50.926380 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf69bb9-8919-4d28-99c6-43af65f0cb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:50.926622 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf69bb9-8919-4d28-99c6-43af65f0cb5b admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:50.927398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf69bb9-8919-4d28-99c6-43af65f0cb5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:50.929340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf69bb9-8919-4d28-99c6-43af65f0cb5b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:51.000143 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1296/2594] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:50 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:51.000352 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67eb6d-f8f0-4208-b240-10469e63e044 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:51.001101 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf69bb9-8919-4d28-99c6-43af65f0cb5b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:51.001853 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-be67eb6d-f8f0-4208-b240-10469e63e044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:51.012276 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf69bb9-8919-4d28-99c6-43af65f0cb5b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:51.015906 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67eb6d-f8f0-4208-b240-10469e63e044 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:51.017611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c43c6-08a4-4d7a-bfe0-c46d24208836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:51.018260 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c43c6-08a4-4d7a-bfe0-c46d24208836 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:51.018537 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c43c6-08a4-4d7a-bfe0-c46d24208836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:51.018807 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c43c6-08a4-4d7a-bfe0-c46d24208836 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:51.019981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c43c6-08a4-4d7a-bfe0-c46d24208836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:51.020990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cca938-b50b-43dd-b968-e617d5d55de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:51.021570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452c43c6-08a4-4d7a-bfe0-c46d24208836 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:51.022005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cca938-b50b-43dd-b968-e617d5d55de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:51.022337 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cca938-b50b-43dd-b968-e617d5d55de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:51.022673 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cca938-b50b-43dd-b968-e617d5d55de8 admin admin] PATH_INFO: `/v3/users/4cca7cfbb1be4d92a603a715af2728f8` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:51.023984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cca938-b50b-43dd-b968-e617d5d55de8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cca7cfbb1be4d92a603a715af2728f8)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:51.025954 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cca938-b50b-43dd-b968-e617d5d55de8 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:51.032428 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1297/2595] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:48:51.036754 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c43c6-08a4-4d7a-bfe0-c46d24208836 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:51.046409 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452c43c6-08a4-4d7a-bfe0-c46d24208836 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:51.052287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f3585-4e5a-40e2-b380-36e9c385b4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:51.053989 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f3585-4e5a-40e2-b380-36e9c385b4f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:51.054284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f3585-4e5a-40e2-b380-36e9c385b4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:51.054566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f3585-4e5a-40e2-b380-36e9c385b4f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:51.055390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f3585-4e5a-40e2-b380-36e9c385b4f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:51.057078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f3585-4e5a-40e2-b380-36e9c385b4f1 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:51.069920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1298/2596] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:51.074026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f3585-4e5a-40e2-b380-36e9c385b4f1 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:51.084132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f3585-4e5a-40e2-b380-36e9c385b4f1 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:51.088703 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecaf40-1494-49ef-abee-2be76d3a6a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:51.089823 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecaf40-1494-49ef-abee-2be76d3a6a06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:51.089823 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecaf40-1494-49ef-abee-2be76d3a6a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:51.090074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecaf40-1494-49ef-abee-2be76d3a6a06 admin admin] PATH_INFO: `/v3/projects/2fd9c389a969423b8e36a41990a82389/users/763acdf1350245d8b2a437e680749939/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:51.104899 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecaf40-1494-49ef-abee-2be76d3a6a06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd9c389a969423b8e36a41990a82389, user_id=763acdf1350245d8b2a437e680749939, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:51.107562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecaf40-1494-49ef-abee-2be76d3a6a06 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:51.202771 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1299/2597] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:51 2022] PUT /identity/v3/projects/2fd9c389a969423b8e36a41990a82389/users/763acdf1350245d8b2a437e680749939/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:51.242582 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-56cca938-b50b-43dd-b968-e617d5d55de8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cca7cfbb1be4d92a603a715af2728f8'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:51.274847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06283ba5-9724-4b62-8baa-1cd6d3b4b161 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:51.274847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06283ba5-9724-4b62-8baa-1cd6d3b4b161 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:51.274847 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06283ba5-9724-4b62-8baa-1cd6d3b4b161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:51.296244 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-06283ba5-9724-4b62-8baa-1cd6d3b4b161 None None] MFA Rules not processed for user `763acdf1350245d8b2a437e680749939`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:51.297443 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-06283ba5-9724-4b62-8baa-1cd6d3b4b161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:51.348166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-56cca938-b50b-43dd-b968-e617d5d55de8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cca7cfbb1be4d92a603a715af2728f8'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:51.356611 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-56cca938-b50b-43dd-b968-e617d5d55de8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cca7cfbb1be4d92a603a715af2728f8'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:51.364020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1300/2598] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:48:51.368597 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06283ba5-9724-4b62-8baa-1cd6d3b4b161 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:51.370751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-56cca938-b50b-43dd-b968-e617d5d55de8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cca7cfbb1be4d92a603a715af2728f8'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:51.371464 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-56cca938-b50b-43dd-b968-e617d5d55de8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cca7cfbb1be4d92a603a715af2728f8'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:51.378169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1299/2599] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:50 2022] DELETE /identity/v3/users/4cca7cfbb1be4d92a603a715af2728f8 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:51.386918 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916aedb-7093-419f-8546-b947a2904ee4 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:51.387298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916aedb-7093-419f-8546-b947a2904ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:51.387583 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916aedb-7093-419f-8546-b947a2904ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:51.405623 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-0916aedb-7093-419f-8546-b947a2904ee4 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:51.407367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-0916aedb-7093-419f-8546-b947a2904ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:51.436537 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06283ba5-9724-4b62-8baa-1cd6d3b4b161 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:51.447807 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:51.462957 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:51.470165 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1300/2600] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:48:51.481354 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916aedb-7093-419f-8546-b947a2904ee4 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:51.483419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-0916aedb-7093-419f-8546-b947a2904ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:51.495224 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bfb45-e716-4c75-a33d-92e8b9957a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:51.498405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bfb45-e716-4c75-a33d-92e8b9957a5b admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:51.498405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bfb45-e716-4c75-a33d-92e8b9957a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:51.498405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bfb45-e716-4c75-a33d-92e8b9957a5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:51.498405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5bfb45-e716-4c75-a33d-92e8b9957a5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:51.498987 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5bfb45-e716-4c75-a33d-92e8b9957a5b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:51.542553 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916aedb-7093-419f-8546-b947a2904ee4 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:51.546194 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe607e9-4899-464e-b930-2e6b24e5884e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:51.546842 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe607e9-4899-464e-b930-2e6b24e5884e service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:51.547160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe607e9-4899-464e-b930-2e6b24e5884e service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:51.547455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe607e9-4899-464e-b930-2e6b24e5884e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:51.557300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe607e9-4899-464e-b930-2e6b24e5884e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:51.558646 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe607e9-4899-464e-b930-2e6b24e5884e service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:51.568354 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1301/2601] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:48:51.589044 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1301/2602] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:51 2022] POST /identity/v3/projects => generated 375 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:48:51.593448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe607e9-4899-464e-b930-2e6b24e5884e service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:51.595933 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-efe607e9-4899-464e-b930-2e6b24e5884e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:51.599273 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bfb45-e716-4c75-a33d-92e8b9957a5b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:51.601055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2b5bfb45-e716-4c75-a33d-92e8b9957a5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:51.612446 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe607e9-4899-464e-b930-2e6b24e5884e service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:51.672870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bfb45-e716-4c75-a33d-92e8b9957a5b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:51.674580 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd7689-fde9-479b-b607-af45eda3161e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:51.675370 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd7689-fde9-479b-b607-af45eda3161e admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:51.675808 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd7689-fde9-479b-b607-af45eda3161e admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:51.676300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd7689-fde9-479b-b607-af45eda3161e admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:51.677030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd7689-fde9-479b-b607-af45eda3161e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:51.678708 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347618a4-7bc5-4caf-b299-c021bbc518e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:51.679017 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd7689-fde9-479b-b607-af45eda3161e admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:51.679946 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347618a4-7bc5-4caf-b299-c021bbc518e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:51.680293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347618a4-7bc5-4caf-b299-c021bbc518e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:51.680688 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347618a4-7bc5-4caf-b299-c021bbc518e7 admin admin] PATH_INFO: `/v3/users/7ffc5e8c234e469d944d9a2cc2795581` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:51.681890 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347618a4-7bc5-4caf-b299-c021bbc518e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ffc5e8c234e469d944d9a2cc2795581)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:51.683840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347618a4-7bc5-4caf-b299-c021bbc518e7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:51.786025 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1302/2603] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:51 2022] POST /identity/v3/users => generated 288 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:48:51.789903 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd7689-fde9-479b-b607-af45eda3161e admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:51.799368 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd7689-fde9-479b-b607-af45eda3161e admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:51.804100 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd8124-5a84-4a95-94b8-46840f2b9a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:51.804780 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd8124-5a84-4a95-94b8-46840f2b9a5d admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:51.805373 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd8124-5a84-4a95-94b8-46840f2b9a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:51.805917 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd8124-5a84-4a95-94b8-46840f2b9a5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:51.807237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd8124-5a84-4a95-94b8-46840f2b9a5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:51.808986 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd8124-5a84-4a95-94b8-46840f2b9a5d admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:51.824583 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1303/2604] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:48:51.824583 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd8124-5a84-4a95-94b8-46840f2b9a5d admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:51.824583 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-8bcd8124-5a84-4a95-94b8-46840f2b9a5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:51.838116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd8124-5a84-4a95-94b8-46840f2b9a5d admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:51.843088 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17634c6-3c53-43a6-8bcf-bc87c7c6a4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:51.844054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17634c6-3c53-43a6-8bcf-bc87c7c6a4cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:51.844216 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17634c6-3c53-43a6-8bcf-bc87c7c6a4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:51.844489 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17634c6-3c53-43a6-8bcf-bc87c7c6a4cf admin admin] PATH_INFO: `/v3/projects/d5231086b3c0494d890eb665eab8d303` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:51.845899 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17634c6-3c53-43a6-8bcf-bc87c7c6a4cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5231086b3c0494d890eb665eab8d303)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:51.848454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17634c6-3c53-43a6-8bcf-bc87c7c6a4cf admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:51.912524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-347618a4-7bc5-4caf-b299-c021bbc518e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ffc5e8c234e469d944d9a2cc2795581'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:51.989377 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c17634c6-3c53-43a6-8bcf-bc87c7c6a4cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5231086b3c0494d890eb665eab8d303', 'request_id': 'req-c17634c6-3c53-43a6-8bcf-bc87c7c6a4cf'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:51.994766 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c17634c6-3c53-43a6-8bcf-bc87c7c6a4cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5231086b3c0494d890eb665eab8d303', 'request_id': 'req-c17634c6-3c53-43a6-8bcf-bc87c7c6a4cf'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:52.001751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c17634c6-3c53-43a6-8bcf-bc87c7c6a4cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5231086b3c0494d890eb665eab8d303', 'request_id': 'req-c17634c6-3c53-43a6-8bcf-bc87c7c6a4cf'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:52.002463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-347618a4-7bc5-4caf-b299-c021bbc518e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ffc5e8c234e469d944d9a2cc2795581'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:52.004513 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-347618a4-7bc5-4caf-b299-c021bbc518e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ffc5e8c234e469d944d9a2cc2795581'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:52.005118 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-347618a4-7bc5-4caf-b299-c021bbc518e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ffc5e8c234e469d944d9a2cc2795581'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:52.012639 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-347618a4-7bc5-4caf-b299-c021bbc518e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ffc5e8c234e469d944d9a2cc2795581'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:52.030619 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1302/2605] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:51 2022] DELETE /identity/v3/users/7ffc5e8c234e469d944d9a2cc2795581 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:52.033705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347618a4-7bc5-4caf-b299-c021bbc518e7 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:52.033705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-347618a4-7bc5-4caf-b299-c021bbc518e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:52.068103 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:52.097055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347618a4-7bc5-4caf-b299-c021bbc518e7 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:52.101797 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb51bf-73c1-4bf0-9913-cd6c8f43efd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:52.102471 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb51bf-73c1-4bf0-9913-cd6c8f43efd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:52.102754 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb51bf-73c1-4bf0-9913-cd6c8f43efd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:52.107645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb51bf-73c1-4bf0-9913-cd6c8f43efd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:52.107645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb51bf-73c1-4bf0-9913-cd6c8f43efd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:52.107885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c17634c6-3c53-43a6-8bcf-bc87c7c6a4cf admin admin] The token cache is being invalidate because project d5231086b3c0494d890eb665eab8d303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:52.107885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c17634c6-3c53-43a6-8bcf-bc87c7c6a4cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:52.107885 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c17634c6-3c53-43a6-8bcf-bc87c7c6a4cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:52.108154 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceb51bf-73c1-4bf0-9913-cd6c8f43efd4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:52.108242 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1304/2606] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:51 2022] DELETE /identity/v3/projects/d5231086b3c0494d890eb665eab8d303 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:52.109538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a21d8-9a8e-453c-bba1-d8263f6c29ca None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:52.109768 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a21d8-9a8e-453c-bba1-d8263f6c29ca None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:52.110051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a21d8-9a8e-453c-bba1-d8263f6c29ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:52.117146 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1303/2607] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:52.126356 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb51bf-73c1-4bf0-9913-cd6c8f43efd4 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:52.128868 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-0ceb51bf-73c1-4bf0-9913-cd6c8f43efd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:52.137411 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-db8a21d8-9a8e-453c-bba1-d8263f6c29ca None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:52.139127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-db8a21d8-9a8e-453c-bba1-d8263f6c29ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:52.143257 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb51bf-73c1-4bf0-9913-cd6c8f43efd4 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:52.147522 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1305/2608] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:52.149681 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bd503-ddd7-4594-ad7c-329a978c083b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:52.150864 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bd503-ddd7-4594-ad7c-329a978c083b admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:52.151198 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bd503-ddd7-4594-ad7c-329a978c083b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:52.151489 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bd503-ddd7-4594-ad7c-329a978c083b admin admin] PATH_INFO: `/v3/projects/8829cb0816b24a1585292916f2876aaf/users/872b8c3a69d14610951c1e81a95d1996/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:52.158730 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a21d8-9a8e-453c-bba1-d8263f6c29ca None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:52.160925 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-db8a21d8-9a8e-453c-bba1-d8263f6c29ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:52.174570 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268bd503-ddd7-4594-ad7c-329a978c083b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8829cb0816b24a1585292916f2876aaf, user_id=872b8c3a69d14610951c1e81a95d1996, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:52.177432 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268bd503-ddd7-4594-ad7c-329a978c083b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:52.208043 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:52.226084 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a21d8-9a8e-453c-bba1-d8263f6c29ca None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:52.229928 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a55006-8b52-4686-b7ad-6fd5e94e48a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:52.230549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a55006-8b52-4686-b7ad-6fd5e94e48a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:52.230843 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a55006-8b52-4686-b7ad-6fd5e94e48a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:52.231152 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a55006-8b52-4686-b7ad-6fd5e94e48a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:52.242429 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a55006-8b52-4686-b7ad-6fd5e94e48a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:52.243969 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a55006-8b52-4686-b7ad-6fd5e94e48a1 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:52.255454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1306/2609] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:48:52.281088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1304/2610] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:52 2022] PUT /identity/v3/projects/8829cb0816b24a1585292916f2876aaf/users/872b8c3a69d14610951c1e81a95d1996/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:52.283932 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de113096-4e1a-4473-a134-e5e03bf50e45 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:52.284227 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de113096-4e1a-4473-a134-e5e03bf50e45 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:52.284587 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de113096-4e1a-4473-a134-e5e03bf50e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:52.316722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-de113096-4e1a-4473-a134-e5e03bf50e45 None None] MFA Rules not processed for user `872b8c3a69d14610951c1e81a95d1996`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:52.318700 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-de113096-4e1a-4473-a134-e5e03bf50e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:52.391531 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1307/2611] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:52.396856 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bd503-ddd7-4594-ad7c-329a978c083b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:52.460224 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bd503-ddd7-4594-ad7c-329a978c083b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:52.465091 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109bcd6-f83e-4474-8e3a-091398e3688a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:52.465753 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109bcd6-f83e-4474-8e3a-091398e3688a admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:52.466035 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109bcd6-f83e-4474-8e3a-091398e3688a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:52.466468 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109bcd6-f83e-4474-8e3a-091398e3688a admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:52.467187 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109bcd6-f83e-4474-8e3a-091398e3688a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:52.469233 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109bcd6-f83e-4474-8e3a-091398e3688a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:52.556184 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1305/2612] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:52 2022] POST /identity/v3/projects => generated 379 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:52.557008 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de113096-4e1a-4473-a134-e5e03bf50e45 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:52.567795 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de113096-4e1a-4473-a134-e5e03bf50e45 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:52.623756 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c601c8-4322-452a-b1c7-dea8aebcf95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:52.626563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c601c8-4322-452a-b1c7-dea8aebcf95b admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:52.626563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c601c8-4322-452a-b1c7-dea8aebcf95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:52.626563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c601c8-4322-452a-b1c7-dea8aebcf95b admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:52.626563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c601c8-4322-452a-b1c7-dea8aebcf95b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:52.627585 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c601c8-4322-452a-b1c7-dea8aebcf95b admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:52.718330 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109bcd6-f83e-4474-8e3a-091398e3688a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:52.720178 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-1109bcd6-f83e-4474-8e3a-091398e3688a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:52.734474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109bcd6-f83e-4474-8e3a-091398e3688a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:52.739551 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a8f68-0f8b-4050-8dca-b8d9acece0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:52.740425 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a8f68-0f8b-4050-8dca-b8d9acece0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:52.740705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a8f68-0f8b-4050-8dca-b8d9acece0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:52.744333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a8f68-0f8b-4050-8dca-b8d9acece0d0 admin admin] PATH_INFO: `/v3/projects/a65fcbc24b0a4f3aba69b909298f267c` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:52.744701 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1308/2613] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:52 2022] POST /identity/v3/users => generated 296 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:48:52.744816 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a8f68-0f8b-4050-8dca-b8d9acece0d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a65fcbc24b0a4f3aba69b909298f267c)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:52.745362 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9a8f68-0f8b-4050-8dca-b8d9acece0d0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:52.746998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c601c8-4322-452a-b1c7-dea8aebcf95b admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:52.757970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c601c8-4322-452a-b1c7-dea8aebcf95b admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:52.762779 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97b121-3458-45f1-a062-1f3e25b85b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:52.763475 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97b121-3458-45f1-a062-1f3e25b85b8c admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:52.763760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97b121-3458-45f1-a062-1f3e25b85b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:52.764038 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97b121-3458-45f1-a062-1f3e25b85b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:52.764767 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97b121-3458-45f1-a062-1f3e25b85b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:52.766347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97b121-3458-45f1-a062-1f3e25b85b8c admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:52.777438 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1309/2614] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:48:52.787476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97b121-3458-45f1-a062-1f3e25b85b8c admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:52.797283 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97b121-3458-45f1-a062-1f3e25b85b8c admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:52.802124 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e2130-bdcb-405a-bbd5-149b63af93c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:52.803020 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e2130-bdcb-405a-bbd5-149b63af93c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:52.803302 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e2130-bdcb-405a-bbd5-149b63af93c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:52.803576 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e2130-bdcb-405a-bbd5-149b63af93c8 admin admin] PATH_INFO: `/v3/projects/da8e51cc09d043ffaa407cba35b1a91b/users/eebce363f4b544cea46e36140653d08b/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:52.818841 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e2130-bdcb-405a-bbd5-149b63af93c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da8e51cc09d043ffaa407cba35b1a91b, user_id=eebce363f4b544cea46e36140653d08b, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:52.821505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e2130-bdcb-405a-bbd5-149b63af93c8 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:52.894586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1310/2615] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:52 2022] PUT /identity/v3/projects/da8e51cc09d043ffaa407cba35b1a91b/users/eebce363f4b544cea46e36140653d08b/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:52.898577 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e2130-bdcb-405a-bbd5-149b63af93c8 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:52.905424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:52.906579 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fb9a8f68-0f8b-4050-8dca-b8d9acece0d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a65fcbc24b0a4f3aba69b909298f267c', 'request_id': 'req-fb9a8f68-0f8b-4050-8dca-b8d9acece0d0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:52.912298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fb9a8f68-0f8b-4050-8dca-b8d9acece0d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a65fcbc24b0a4f3aba69b909298f267c', 'request_id': 'req-fb9a8f68-0f8b-4050-8dca-b8d9acece0d0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:52.919185 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fb9a8f68-0f8b-4050-8dca-b8d9acece0d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a65fcbc24b0a4f3aba69b909298f267c', 'request_id': 'req-fb9a8f68-0f8b-4050-8dca-b8d9acece0d0'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:52.971719 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e2130-bdcb-405a-bbd5-149b63af93c8 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:53.006586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fb9a8f68-0f8b-4050-8dca-b8d9acece0d0 admin admin] The token cache is being invalidate because project a65fcbc24b0a4f3aba69b909298f267c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:53.007020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fb9a8f68-0f8b-4050-8dca-b8d9acece0d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:53.007682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fb9a8f68-0f8b-4050-8dca-b8d9acece0d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:53.009242 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1306/2616] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:52 2022] DELETE /identity/v3/projects/a65fcbc24b0a4f3aba69b909298f267c => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:53.013103 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a8f68-0f8b-4050-8dca-b8d9acece0d0 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:53.014465 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-fb9a8f68-0f8b-4050-8dca-b8d9acece0d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:53.029031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:53.032911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c597ae9-2fdc-46f2-a67f-bb7015f3fb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:53.032911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c597ae9-2fdc-46f2-a67f-bb7015f3fb68 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:53.032911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c597ae9-2fdc-46f2-a67f-bb7015f3fb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:53.032911 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c597ae9-2fdc-46f2-a67f-bb7015f3fb68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:53.033502 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c597ae9-2fdc-46f2-a67f-bb7015f3fb68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:53.035144 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c597ae9-2fdc-46f2-a67f-bb7015f3fb68 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:53.045131 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1311/2617] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:48:53.049175 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c597ae9-2fdc-46f2-a67f-bb7015f3fb68 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:53.051470 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-9c597ae9-2fdc-46f2-a67f-bb7015f3fb68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:53.063794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c597ae9-2fdc-46f2-a67f-bb7015f3fb68 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:53.069111 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc3e9e3-ad10-4def-9644-baa52da1ef45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:53.069986 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3e9e3-ad10-4def-9644-baa52da1ef45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:53.070272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3e9e3-ad10-4def-9644-baa52da1ef45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:53.070552 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc3e9e3-ad10-4def-9644-baa52da1ef45 admin admin] PATH_INFO: `/v3/projects/da8e51cc09d043ffaa407cba35b1a91b/users/eebce363f4b544cea46e36140653d08b/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:53.072973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc3e9e3-ad10-4def-9644-baa52da1ef45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da8e51cc09d043ffaa407cba35b1a91b, user_id=eebce363f4b544cea46e36140653d08b, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:53.075621 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc3e9e3-ad10-4def-9644-baa52da1ef45 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:53.082434 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a8f68-0f8b-4050-8dca-b8d9acece0d0 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:53.085863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1312/2618] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:53 2022] PUT /identity/v3/projects/da8e51cc09d043ffaa407cba35b1a91b/users/eebce363f4b544cea46e36140653d08b/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:53.097184 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c6bd1-4137-4b4b-a9a7-78e4aa67f3f5 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:53.097497 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c6bd1-4137-4b4b-a9a7-78e4aa67f3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:53.097778 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c6bd1-4137-4b4b-a9a7-78e4aa67f3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:53.123740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-805c6bd1-4137-4b4b-a9a7-78e4aa67f3f5 None None] MFA Rules not processed for user `eebce363f4b544cea46e36140653d08b`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:53.124933 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-805c6bd1-4137-4b4b-a9a7-78e4aa67f3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:53.140611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622dc145-6d23-4c56-895b-0c0837a0e158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:53.141582 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dc145-6d23-4c56-895b-0c0837a0e158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:53.141885 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dc145-6d23-4c56-895b-0c0837a0e158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:53.142159 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dc145-6d23-4c56-895b-0c0837a0e158 admin admin] PATH_INFO: `/v3/users/a2d8309114ef416595b218dcc37331d4` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:53.143396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622dc145-6d23-4c56-895b-0c0837a0e158 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2d8309114ef416595b218dcc37331d4)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:53.145527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622dc145-6d23-4c56-895b-0c0837a0e158 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:53.194744 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1313/2619] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:48:53.205659 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c6bd1-4137-4b4b-a9a7-78e4aa67f3f5 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:53.207070 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-805c6bd1-4137-4b4b-a9a7-78e4aa67f3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:53.263272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c6bd1-4137-4b4b-a9a7-78e4aa67f3f5 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:53.266992 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5206496-39a6-426c-a1ed-149aa80eba72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:53.267666 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5206496-39a6-426c-a1ed-149aa80eba72 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:53.267951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5206496-39a6-426c-a1ed-149aa80eba72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:53.268229 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5206496-39a6-426c-a1ed-149aa80eba72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:53.277790 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5206496-39a6-426c-a1ed-149aa80eba72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:53.281460 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5206496-39a6-426c-a1ed-149aa80eba72 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:53.288963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1314/2620] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:53 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:48:53.293858 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:53.308243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-622dc145-6d23-4c56-895b-0c0837a0e158 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2d8309114ef416595b218dcc37331d4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:53.402512 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-622dc145-6d23-4c56-895b-0c0837a0e158 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2d8309114ef416595b218dcc37331d4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:53.403119 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-622dc145-6d23-4c56-895b-0c0837a0e158 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2d8309114ef416595b218dcc37331d4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:53.403747 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-622dc145-6d23-4c56-895b-0c0837a0e158 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2d8309114ef416595b218dcc37331d4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:53.412066 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-622dc145-6d23-4c56-895b-0c0837a0e158 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2d8309114ef416595b218dcc37331d4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:53.422527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5206496-39a6-426c-a1ed-149aa80eba72 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:53.423974 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-d5206496-39a6-426c-a1ed-149aa80eba72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:53.430510 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1307/2621] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:53 2022] DELETE /identity/v3/users/a2d8309114ef416595b218dcc37331d4 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:53.436103 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5206496-39a6-426c-a1ed-149aa80eba72 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:53.485728 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059dcb2a-dce0-4555-bb7f-ab11b3940900 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:53.486380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dcb2a-dce0-4555-bb7f-ab11b3940900 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:53.486656 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dcb2a-dce0-4555-bb7f-ab11b3940900 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:53.486954 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059dcb2a-dce0-4555-bb7f-ab11b3940900 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:53.489552 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-059dcb2a-dce0-4555-bb7f-ab11b3940900 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:53.553526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059dcb2a-dce0-4555-bb7f-ab11b3940900 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:53.554995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059dcb2a-dce0-4555-bb7f-ab11b3940900 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:53.563960 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1315/2622] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:48:53.869316 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622dc145-6d23-4c56-895b-0c0837a0e158 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:53.869316 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-622dc145-6d23-4c56-895b-0c0837a0e158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:53.918085 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:53.939464 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622dc145-6d23-4c56-895b-0c0837a0e158 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:53.944022 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea2c21-38fb-4971-8011-acb980bdf4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:53.944882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea2c21-38fb-4971-8011-acb980bdf4bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:53.945164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea2c21-38fb-4971-8011-acb980bdf4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:53.945437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea2c21-38fb-4971-8011-acb980bdf4bc admin admin] PATH_INFO: `/v3/projects/fd6d3810bcc14bb694f8f9b43325bd7e` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:53.946626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea2c21-38fb-4971-8011-acb980bdf4bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd6d3810bcc14bb694f8f9b43325bd7e)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:53.948580 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea2c21-38fb-4971-8011-acb980bdf4bc admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:54.052154 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-98ea2c21-38fb-4971-8011-acb980bdf4bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd6d3810bcc14bb694f8f9b43325bd7e', 'request_id': 'req-98ea2c21-38fb-4971-8011-acb980bdf4bc'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:54.057662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-98ea2c21-38fb-4971-8011-acb980bdf4bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd6d3810bcc14bb694f8f9b43325bd7e', 'request_id': 'req-98ea2c21-38fb-4971-8011-acb980bdf4bc'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:54.063906 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-98ea2c21-38fb-4971-8011-acb980bdf4bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd6d3810bcc14bb694f8f9b43325bd7e', 'request_id': 'req-98ea2c21-38fb-4971-8011-acb980bdf4bc'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:54.145097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-98ea2c21-38fb-4971-8011-acb980bdf4bc admin admin] The token cache is being invalidate because project fd6d3810bcc14bb694f8f9b43325bd7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:54.145382 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-98ea2c21-38fb-4971-8011-acb980bdf4bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:54.146157 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-98ea2c21-38fb-4971-8011-acb980bdf4bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:54.155564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1308/2623] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:53 2022] DELETE /identity/v3/projects/fd6d3810bcc14bb694f8f9b43325bd7e => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:54.155713 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059dcb2a-dce0-4555-bb7f-ab11b3940900 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:54.163049 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059dcb2a-dce0-4555-bb7f-ab11b3940900 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:54.201480 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:54.219733 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd198d-e0c1-4147-b629-7cfefa44b3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:54.220925 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd198d-e0c1-4147-b629-7cfefa44b3f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:54.221203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd198d-e0c1-4147-b629-7cfefa44b3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:54.221470 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd198d-e0c1-4147-b629-7cfefa44b3f2 admin admin] PATH_INFO: `/v3/users/22dcb67ff7eb443fb8fd6f036d3d2e7a` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:54.222684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd198d-e0c1-4147-b629-7cfefa44b3f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22dcb67ff7eb443fb8fd6f036d3d2e7a)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:54.224713 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd198d-e0c1-4147-b629-7cfefa44b3f2 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:54.439742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-21bd198d-e0c1-4147-b629-7cfefa44b3f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22dcb67ff7eb443fb8fd6f036d3d2e7a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:54.573605 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-21bd198d-e0c1-4147-b629-7cfefa44b3f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22dcb67ff7eb443fb8fd6f036d3d2e7a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:54.581241 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-21bd198d-e0c1-4147-b629-7cfefa44b3f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22dcb67ff7eb443fb8fd6f036d3d2e7a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:54.593344 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-21bd198d-e0c1-4147-b629-7cfefa44b3f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22dcb67ff7eb443fb8fd6f036d3d2e7a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:54.593979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-21bd198d-e0c1-4147-b629-7cfefa44b3f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22dcb67ff7eb443fb8fd6f036d3d2e7a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:54.600384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1316/2624] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:54 2022] DELETE /identity/v3/users/22dcb67ff7eb443fb8fd6f036d3d2e7a => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:55.026353 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea2c21-38fb-4971-8011-acb980bdf4bc admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:55.027807 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-98ea2c21-38fb-4971-8011-acb980bdf4bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:55.056783 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea2c21-38fb-4971-8011-acb980bdf4bc admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:55.115120 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b3cba1-49e7-4e4f-8fb7-d74727687263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:55.115987 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3cba1-49e7-4e4f-8fb7-d74727687263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:55.116269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3cba1-49e7-4e4f-8fb7-d74727687263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:55.116545 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3cba1-49e7-4e4f-8fb7-d74727687263 admin admin] PATH_INFO: `/v3/projects/78a94d4412b546c9abc56a6ea233a4d9` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:55.117719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b3cba1-49e7-4e4f-8fb7-d74727687263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78a94d4412b546c9abc56a6ea233a4d9)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:55.119696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b3cba1-49e7-4e4f-8fb7-d74727687263 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:55.214508 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d0b3cba1-49e7-4e4f-8fb7-d74727687263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78a94d4412b546c9abc56a6ea233a4d9', 'request_id': 'req-d0b3cba1-49e7-4e4f-8fb7-d74727687263'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:55.219945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d0b3cba1-49e7-4e4f-8fb7-d74727687263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78a94d4412b546c9abc56a6ea233a4d9', 'request_id': 'req-d0b3cba1-49e7-4e4f-8fb7-d74727687263'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:55.227060 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d0b3cba1-49e7-4e4f-8fb7-d74727687263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78a94d4412b546c9abc56a6ea233a4d9', 'request_id': 'req-d0b3cba1-49e7-4e4f-8fb7-d74727687263'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:55.323934 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:55.328344 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d0b3cba1-49e7-4e4f-8fb7-d74727687263 admin admin] The token cache is being invalidate because project 78a94d4412b546c9abc56a6ea233a4d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:55.328748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d0b3cba1-49e7-4e4f-8fb7-d74727687263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:55.329373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-d0b3cba1-49e7-4e4f-8fb7-d74727687263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:55.330930 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1309/2625] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:55 2022] DELETE /identity/v3/projects/78a94d4412b546c9abc56a6ea233a4d9 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:55.337593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c9ec2-91fb-41eb-a17d-b6c5d551b282 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:55.337920 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c9ec2-91fb-41eb-a17d-b6c5d551b282 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:55.338200 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c9ec2-91fb-41eb-a17d-b6c5d551b282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:55.358704 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-cb2c9ec2-91fb-41eb-a17d-b6c5d551b282 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:55.360045 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-cb2c9ec2-91fb-41eb-a17d-b6c5d551b282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:55.419455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1317/2626] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:48:55.432311 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b0240-cbe3-48fe-987c-5dd0b2993bb7 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:55.432602 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b0240-cbe3-48fe-987c-5dd0b2993bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:55.432884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53b0240-cbe3-48fe-987c-5dd0b2993bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:55.452107 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a53b0240-cbe3-48fe-987c-5dd0b2993bb7 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:55.455242 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a53b0240-cbe3-48fe-987c-5dd0b2993bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:55.462833 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1310/2627] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:55.467208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c9ec2-91fb-41eb-a17d-b6c5d551b282 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:55.477775 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c9ec2-91fb-41eb-a17d-b6c5d551b282 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:55.482414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e519b14-6f39-4ef2-acb0-9f4520b4a3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:55.483100 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e519b14-6f39-4ef2-acb0-9f4520b4a3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:55.483386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e519b14-6f39-4ef2-acb0-9f4520b4a3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:55.483662 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e519b14-6f39-4ef2-acb0-9f4520b4a3a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:55.484389 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e519b14-6f39-4ef2-acb0-9f4520b4a3a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:48:55.484696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e519b14-6f39-4ef2-acb0-9f4520b4a3a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:55.486279 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e519b14-6f39-4ef2-acb0-9f4520b4a3a1 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:55.497953 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1318/2628] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:48:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:48:55.512087 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b0240-cbe3-48fe-987c-5dd0b2993bb7 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:55.522561 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53b0240-cbe3-48fe-987c-5dd0b2993bb7 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:55.527676 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c3ec1-5908-4c09-ab35-9b236486f24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:55.528431 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c3ec1-5908-4c09-ab35-9b236486f24b admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:55.529051 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c3ec1-5908-4c09-ab35-9b236486f24b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:55.529051 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c3ec1-5908-4c09-ab35-9b236486f24b admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:55.530527 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c3ec1-5908-4c09-ab35-9b236486f24b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:55.532011 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c3ec1-5908-4c09-ab35-9b236486f24b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:55.648291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1311/2629] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:55 2022] POST /identity/v3/projects => generated 381 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:48:55.652668 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e519b14-6f39-4ef2-acb0-9f4520b4a3a1 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:55.663321 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e519b14-6f39-4ef2-acb0-9f4520b4a3a1 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:55.710105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:55.722595 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe19fb4-0986-4680-89d9-666cc3da9f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:55.723398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe19fb4-0986-4680-89d9-666cc3da9f1c admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:55.727067 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe19fb4-0986-4680-89d9-666cc3da9f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:55.727067 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe19fb4-0986-4680-89d9-666cc3da9f1c admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:55.727067 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe19fb4-0986-4680-89d9-666cc3da9f1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:55.727067 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe19fb4-0986-4680-89d9-666cc3da9f1c admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:55.825284 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1319/2630] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:55 2022] POST /identity/v3/users => generated 291 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:55.829493 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c3ec1-5908-4c09-ab35-9b236486f24b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:55.840417 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c3ec1-5908-4c09-ab35-9b236486f24b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:55.845039 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7529d-9015-48eb-b3bb-82431bc90190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:55.845694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7529d-9015-48eb-b3bb-82431bc90190 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:55.845982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7529d-9015-48eb-b3bb-82431bc90190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:55.846250 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7529d-9015-48eb-b3bb-82431bc90190 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:55.847050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7529d-9015-48eb-b3bb-82431bc90190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:55.848722 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7529d-9015-48eb-b3bb-82431bc90190 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:55.858743 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1312/2631] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:55.867984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe19fb4-0986-4680-89d9-666cc3da9f1c admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:55.878151 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe19fb4-0986-4680-89d9-666cc3da9f1c admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:55.883383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac86741-c239-4f31-96d4-d09425cc2d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:55.884095 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac86741-c239-4f31-96d4-d09425cc2d3b admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:55.884732 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac86741-c239-4f31-96d4-d09425cc2d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:55.885017 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac86741-c239-4f31-96d4-d09425cc2d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:55.885735 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac86741-c239-4f31-96d4-d09425cc2d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:55.887537 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac86741-c239-4f31-96d4-d09425cc2d3b admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:55.904608 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1320/2632] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:55.909165 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7529d-9015-48eb-b3bb-82431bc90190 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:55.919272 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7529d-9015-48eb-b3bb-82431bc90190 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:55.924171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076d31a-95cb-41e8-8beb-a5ed8c5fbaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:55.925190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076d31a-95cb-41e8-8beb-a5ed8c5fbaec admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:55.925667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076d31a-95cb-41e8-8beb-a5ed8c5fbaec admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:55.925952 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076d31a-95cb-41e8-8beb-a5ed8c5fbaec admin admin] PATH_INFO: `/v3/projects/3cabb5fae51f4b74b45ae11c0a1d46cf/users/e8f6424286c8446ca2b4bbb18e04fee8/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:55.940450 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076d31a-95cb-41e8-8beb-a5ed8c5fbaec admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cabb5fae51f4b74b45ae11c0a1d46cf, user_id=e8f6424286c8446ca2b4bbb18e04fee8, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:55.943692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076d31a-95cb-41e8-8beb-a5ed8c5fbaec admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:56.009055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1313/2633] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:55 2022] PUT /identity/v3/projects/3cabb5fae51f4b74b45ae11c0a1d46cf/users/e8f6424286c8446ca2b4bbb18e04fee8/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:56.021266 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa7091-6f34-4eb6-a605-c87fb1b5bd0e None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:56.021632 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa7091-6f34-4eb6-a605-c87fb1b5bd0e None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:56.021897 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa7091-6f34-4eb6-a605-c87fb1b5bd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:56.049345 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-6caa7091-6f34-4eb6-a605-c87fb1b5bd0e None None] MFA Rules not processed for user `e8f6424286c8446ca2b4bbb18e04fee8`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:56.051214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-6caa7091-6f34-4eb6-a605-c87fb1b5bd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:56.119768 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1321/2634] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:56.124384 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076d31a-95cb-41e8-8beb-a5ed8c5fbaec admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:56.188927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076d31a-95cb-41e8-8beb-a5ed8c5fbaec admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:56.194394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6cf4ba-95e9-4b96-9e4e-9fc40f4f3d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:56.195092 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cf4ba-95e9-4b96-9e4e-9fc40f4f3d05 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:56.195381 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cf4ba-95e9-4b96-9e4e-9fc40f4f3d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:56.195659 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cf4ba-95e9-4b96-9e4e-9fc40f4f3d05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:56.196355 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6cf4ba-95e9-4b96-9e4e-9fc40f4f3d05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:56.198255 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6cf4ba-95e9-4b96-9e4e-9fc40f4f3d05 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:56.311884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1314/2635] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:48:56 2022] POST /identity/v3/projects => generated 379 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:48:56.316588 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa7091-6f34-4eb6-a605-c87fb1b5bd0e None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:56.330232 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa7091-6f34-4eb6-a605-c87fb1b5bd0e None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:56.383998 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc708c6-b05f-4c1f-98b5-878777a2bbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:56.384736 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc708c6-b05f-4c1f-98b5-878777a2bbcd admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:56.385019 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc708c6-b05f-4c1f-98b5-878777a2bbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:56.385292 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc708c6-b05f-4c1f-98b5-878777a2bbcd admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:56.385980 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc708c6-b05f-4c1f-98b5-878777a2bbcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:56.388039 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc708c6-b05f-4c1f-98b5-878777a2bbcd admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:56.469268 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1322/2636] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:48:56 2022] POST /identity/v3/users => generated 296 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:48:56.473921 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6cf4ba-95e9-4b96-9e4e-9fc40f4f3d05 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:56.484416 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6cf4ba-95e9-4b96-9e4e-9fc40f4f3d05 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:56.489851 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741f2d2-f8c2-4202-a4a1-9de673e3039d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:56.490592 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741f2d2-f8c2-4202-a4a1-9de673e3039d admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:56.490872 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741f2d2-f8c2-4202-a4a1-9de673e3039d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:56.491171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741f2d2-f8c2-4202-a4a1-9de673e3039d admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:56.491885 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741f2d2-f8c2-4202-a4a1-9de673e3039d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:56.493468 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741f2d2-f8c2-4202-a4a1-9de673e3039d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:56.504185 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1315/2637] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:48:56.508410 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc708c6-b05f-4c1f-98b5-878777a2bbcd admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:56.518316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc708c6-b05f-4c1f-98b5-878777a2bbcd admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:56.522984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39e4b2-8342-48bf-84fc-7a3abaf3b42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:56.523853 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39e4b2-8342-48bf-84fc-7a3abaf3b42a admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:56.524133 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39e4b2-8342-48bf-84fc-7a3abaf3b42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:56.524425 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39e4b2-8342-48bf-84fc-7a3abaf3b42a admin admin] PATH_INFO: `/v3/projects/c4ea07a3c2264398a3d6b232ea250fea/users/1e2686e18e184f0f8d08f0c686c30698/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:56.540016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc39e4b2-8342-48bf-84fc-7a3abaf3b42a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4ea07a3c2264398a3d6b232ea250fea, user_id=1e2686e18e184f0f8d08f0c686c30698, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:56.542809 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc39e4b2-8342-48bf-84fc-7a3abaf3b42a admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:56.636328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1323/2638] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:56 2022] PUT /identity/v3/projects/c4ea07a3c2264398a3d6b232ea250fea/users/1e2686e18e184f0f8d08f0c686c30698/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:56.640421 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741f2d2-f8c2-4202-a4a1-9de673e3039d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:56.650604 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741f2d2-f8c2-4202-a4a1-9de673e3039d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:56.707763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502facb-06c1-4e78-82cb-ff82413b8108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:56.708430 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502facb-06c1-4e78-82cb-ff82413b8108 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:56.708712 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502facb-06c1-4e78-82cb-ff82413b8108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:56.709042 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502facb-06c1-4e78-82cb-ff82413b8108 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:56.709761 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502facb-06c1-4e78-82cb-ff82413b8108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:56.711499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502facb-06c1-4e78-82cb-ff82413b8108 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:56.721121 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1316/2639] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:48:56.726036 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39e4b2-8342-48bf-84fc-7a3abaf3b42a admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:56.736439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39e4b2-8342-48bf-84fc-7a3abaf3b42a admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:56.741378 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c375c-f45e-47d0-9131-eb23f63092fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:56.742251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c375c-f45e-47d0-9131-eb23f63092fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:56.742528 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c375c-f45e-47d0-9131-eb23f63092fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:56.742848 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c375c-f45e-47d0-9131-eb23f63092fe admin admin] PATH_INFO: `/v3/projects/c4ea07a3c2264398a3d6b232ea250fea/users/1e2686e18e184f0f8d08f0c686c30698/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:56.744865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c375c-f45e-47d0-9131-eb23f63092fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4ea07a3c2264398a3d6b232ea250fea, user_id=1e2686e18e184f0f8d08f0c686c30698, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:56.747647 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c375c-f45e-47d0-9131-eb23f63092fe admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:56.758651 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1324/2640] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:48:56 2022] PUT /identity/v3/projects/c4ea07a3c2264398a3d6b232ea250fea/users/1e2686e18e184f0f8d08f0c686c30698/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:56.770347 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7a15d-a927-4be8-b98b-3dc445d35dba None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:56.770641 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7a15d-a927-4be8-b98b-3dc445d35dba None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:56.770959 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7a15d-a927-4be8-b98b-3dc445d35dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:56.783941 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:56.798771 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-40f7a15d-a927-4be8-b98b-3dc445d35dba None None] MFA Rules not processed for user `1e2686e18e184f0f8d08f0c686c30698`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:56.800076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-40f7a15d-a927-4be8-b98b-3dc445d35dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:56.871232 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1317/2641] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:56 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:48:56.882592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c375c-f45e-47d0-9131-eb23f63092fe admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:56.887660 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-d39c375c-f45e-47d0-9131-eb23f63092fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:56.895243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:56.944739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c375c-f45e-47d0-9131-eb23f63092fe admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:56.948457 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c3c40-e478-4615-a537-aa6a8d16d351 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:56.949100 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c3c40-e478-4615-a537-aa6a8d16d351 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:56.949382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c3c40-e478-4615-a537-aa6a8d16d351 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:56.949658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c3c40-e478-4615-a537-aa6a8d16d351 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:56.959583 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c3c40-e478-4615-a537-aa6a8d16d351 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:56.961016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c3c40-e478-4615-a537-aa6a8d16d351 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:56.970946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1325/2642] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:56 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:48:57.475747 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7a15d-a927-4be8-b98b-3dc445d35dba None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:57.539691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7a15d-a927-4be8-b98b-3dc445d35dba None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:57.544192 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfce54-65ab-440f-b6c3-8cd4b005d66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:57.545055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfce54-65ab-440f-b6c3-8cd4b005d66f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:57.545334 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfce54-65ab-440f-b6c3-8cd4b005d66f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:57.545607 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfce54-65ab-440f-b6c3-8cd4b005d66f admin admin] PATH_INFO: `/v3/users/e8f6424286c8446ca2b4bbb18e04fee8` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:57.546822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfce54-65ab-440f-b6c3-8cd4b005d66f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8f6424286c8446ca2b4bbb18e04fee8)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:57.548731 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edfce54-65ab-440f-b6c3-8cd4b005d66f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:57.586846 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c3c40-e478-4615-a537-aa6a8d16d351 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:57.588388 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-2a6c3c40-e478-4615-a537-aa6a8d16d351 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:57.602000 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c3c40-e478-4615-a537-aa6a8d16d351 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:57.606410 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f36f8-fbe5-47a3-8769-09f6f078e6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:57.607285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f36f8-fbe5-47a3-8769-09f6f078e6d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:57.611208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f36f8-fbe5-47a3-8769-09f6f078e6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:57.611208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f36f8-fbe5-47a3-8769-09f6f078e6d4 admin admin] PATH_INFO: `/v3/users/763acdf1350245d8b2a437e680749939` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:57.611208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f36f8-fbe5-47a3-8769-09f6f078e6d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=763acdf1350245d8b2a437e680749939)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:57.611208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f36f8-fbe5-47a3-8769-09f6f078e6d4 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:57.769729 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3edfce54-65ab-440f-b6c3-8cd4b005d66f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8f6424286c8446ca2b4bbb18e04fee8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:57.827820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-235f36f8-fbe5-47a3-8769-09f6f078e6d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '763acdf1350245d8b2a437e680749939'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:57.841995 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3edfce54-65ab-440f-b6c3-8cd4b005d66f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8f6424286c8446ca2b4bbb18e04fee8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:57.842623 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3edfce54-65ab-440f-b6c3-8cd4b005d66f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8f6424286c8446ca2b4bbb18e04fee8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:57.843251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3edfce54-65ab-440f-b6c3-8cd4b005d66f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8f6424286c8446ca2b4bbb18e04fee8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:57.850977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3edfce54-65ab-440f-b6c3-8cd4b005d66f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8f6424286c8446ca2b4bbb18e04fee8'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:57.868785 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1318/2643] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:57 2022] DELETE /identity/v3/users/e8f6424286c8446ca2b4bbb18e04fee8 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:57.873114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee50765-881f-405b-b38f-3b3140c0fb99 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:57.873401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee50765-881f-405b-b38f-3b3140c0fb99 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:57.873676 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee50765-881f-405b-b38f-3b3140c0fb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:57.891109 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-eee50765-881f-405b-b38f-3b3140c0fb99 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:57.892330 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-eee50765-881f-405b-b38f-3b3140c0fb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:57.897610 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:57.911292 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-235f36f8-fbe5-47a3-8769-09f6f078e6d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '763acdf1350245d8b2a437e680749939'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:57.919267 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-235f36f8-fbe5-47a3-8769-09f6f078e6d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '763acdf1350245d8b2a437e680749939'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:57.933541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-235f36f8-fbe5-47a3-8769-09f6f078e6d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '763acdf1350245d8b2a437e680749939'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:57.934259 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-235f36f8-fbe5-47a3-8769-09f6f078e6d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '763acdf1350245d8b2a437e680749939'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:57.940396 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1326/2644] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:57 2022] DELETE /identity/v3/users/763acdf1350245d8b2a437e680749939 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:57.944986 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b69e92-1cf3-447e-a914-47979560577b None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:57.945275 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b69e92-1cf3-447e-a914-47979560577b None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:57.945553 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b69e92-1cf3-447e-a914-47979560577b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:57.955704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1319/2645] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:48:57.963760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-91b69e92-1cf3-447e-a914-47979560577b None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:48:57.965002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-91b69e92-1cf3-447e-a914-47979560577b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:57.966476 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee50765-881f-405b-b38f-3b3140c0fb99 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:57.967742 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-eee50765-881f-405b-b38f-3b3140c0fb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:58.024534 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1327/2646] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:48:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:48:58.032606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee50765-881f-405b-b38f-3b3140c0fb99 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:58.035132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b69e92-1cf3-447e-a914-47979560577b None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:58.061001 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:58.082733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade37d9-e34b-4019-b891-62b2836caea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:58.083423 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade37d9-e34b-4019-b891-62b2836caea9 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:58.083708 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade37d9-e34b-4019-b891-62b2836caea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:58.084078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade37d9-e34b-4019-b891-62b2836caea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:58.090516 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b69e92-1cf3-447e-a914-47979560577b None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:58.094154 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4bb95-0816-43f4-b120-01fe8d45f2d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:58.094692 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade37d9-e34b-4019-b891-62b2836caea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:58.094809 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4bb95-0816-43f4-b120-01fe8d45f2d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:58.095093 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4bb95-0816-43f4-b120-01fe8d45f2d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:58.095371 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4bb95-0816-43f4-b120-01fe8d45f2d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:58.096153 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade37d9-e34b-4019-b891-62b2836caea9 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:58.105334 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd4bb95-0816-43f4-b120-01fe8d45f2d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:58.106577 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1320/2647] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:48:58.111303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd4bb95-0816-43f4-b120-01fe8d45f2d3 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:58.122316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1328/2648] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:48:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:48:58.744484 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade37d9-e34b-4019-b891-62b2836caea9 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:58.746036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-7ade37d9-e34b-4019-b891-62b2836caea9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:58.762385 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade37d9-e34b-4019-b891-62b2836caea9 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:58.767903 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e1c02-6d52-4fa1-a708-5eef315d5e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:58.768803 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e1c02-6d52-4fa1-a708-5eef315d5e14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:58.769713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e1c02-6d52-4fa1-a708-5eef315d5e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:58.769947 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e1c02-6d52-4fa1-a708-5eef315d5e14 admin admin] PATH_INFO: `/v3/projects/3cabb5fae51f4b74b45ae11c0a1d46cf` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:58.771138 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e1c02-6d52-4fa1-a708-5eef315d5e14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cabb5fae51f4b74b45ae11c0a1d46cf)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:58.773256 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e1c02-6d52-4fa1-a708-5eef315d5e14 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:58.787093 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4bb95-0816-43f4-b120-01fe8d45f2d3 service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:58.790775 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-acd4bb95-0816-43f4-b120-01fe8d45f2d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:58.802346 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4bb95-0816-43f4-b120-01fe8d45f2d3 service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:58.807592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f60bef3-4c34-4aab-bddc-122687e61a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:58.808516 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60bef3-4c34-4aab-bddc-122687e61a63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:58.813830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60bef3-4c34-4aab-bddc-122687e61a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:58.813830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60bef3-4c34-4aab-bddc-122687e61a63 admin admin] PATH_INFO: `/v3/projects/2fd9c389a969423b8e36a41990a82389` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:58.813830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f60bef3-4c34-4aab-bddc-122687e61a63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fd9c389a969423b8e36a41990a82389)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:58.813830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f60bef3-4c34-4aab-bddc-122687e61a63 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:58.900188 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-036e1c02-6d52-4fa1-a708-5eef315d5e14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cabb5fae51f4b74b45ae11c0a1d46cf', 'request_id': 'req-036e1c02-6d52-4fa1-a708-5eef315d5e14'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:58.906586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-036e1c02-6d52-4fa1-a708-5eef315d5e14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cabb5fae51f4b74b45ae11c0a1d46cf', 'request_id': 'req-036e1c02-6d52-4fa1-a708-5eef315d5e14'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:58.912968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-036e1c02-6d52-4fa1-a708-5eef315d5e14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cabb5fae51f4b74b45ae11c0a1d46cf', 'request_id': 'req-036e1c02-6d52-4fa1-a708-5eef315d5e14'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:58.924536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:58.928348 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-8f60bef3-4c34-4aab-bddc-122687e61a63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fd9c389a969423b8e36a41990a82389', 'request_id': 'req-8f60bef3-4c34-4aab-bddc-122687e61a63'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:58.933921 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-8f60bef3-4c34-4aab-bddc-122687e61a63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fd9c389a969423b8e36a41990a82389', 'request_id': 'req-8f60bef3-4c34-4aab-bddc-122687e61a63'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:58.941247 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-8f60bef3-4c34-4aab-bddc-122687e61a63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fd9c389a969423b8e36a41990a82389', 'request_id': 'req-8f60bef3-4c34-4aab-bddc-122687e61a63'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:59.024238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-036e1c02-6d52-4fa1-a708-5eef315d5e14 admin admin] The token cache is being invalidate because project 3cabb5fae51f4b74b45ae11c0a1d46cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:59.024534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-036e1c02-6d52-4fa1-a708-5eef315d5e14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:59.025158 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-036e1c02-6d52-4fa1-a708-5eef315d5e14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:59.026701 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1321/2649] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:58 2022] DELETE /identity/v3/projects/3cabb5fae51f4b74b45ae11c0a1d46cf => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:48:59.030843 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e1c02-6d52-4fa1-a708-5eef315d5e14 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:59.032177 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-036e1c02-6d52-4fa1-a708-5eef315d5e14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:59.070799 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-8f60bef3-4c34-4aab-bddc-122687e61a63 admin admin] The token cache is being invalidate because project 2fd9c389a969423b8e36a41990a82389 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:48:59.071052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-8f60bef3-4c34-4aab-bddc-122687e61a63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:59.071690 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-8f60bef3-4c34-4aab-bddc-122687e61a63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:59.073228 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1329/2650] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:48:58 2022] DELETE /identity/v3/projects/2fd9c389a969423b8e36a41990a82389 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:59.077195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f60bef3-4c34-4aab-bddc-122687e61a63 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:48:59.080032 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-8f60bef3-4c34-4aab-bddc-122687e61a63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:48:59.099254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e1c02-6d52-4fa1-a708-5eef315d5e14 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:59.146148 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f60bef3-4c34-4aab-bddc-122687e61a63 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:48:59.150878 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621b703-3c51-4154-9dd0-951c0c8a9d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:59.151782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621b703-3c51-4154-9dd0-951c0c8a9d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:59.152065 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621b703-3c51-4154-9dd0-951c0c8a9d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:59.152339 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621b703-3c51-4154-9dd0-951c0c8a9d1b admin admin] PATH_INFO: `/v3/users/872b8c3a69d14610951c1e81a95d1996` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:59.153579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621b703-3c51-4154-9dd0-951c0c8a9d1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=872b8c3a69d14610951c1e81a95d1996)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:59.155498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb505fc3-d5b0-41a3-ac0b-b6aac28817b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:48:59.155863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621b703-3c51-4154-9dd0-951c0c8a9d1b admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:59.156368 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb505fc3-d5b0-41a3-ac0b-b6aac28817b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:48:59.156646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb505fc3-d5b0-41a3-ac0b-b6aac28817b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:48:59.156914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb505fc3-d5b0-41a3-ac0b-b6aac28817b2 admin admin] PATH_INFO: `/v3/users/1e2686e18e184f0f8d08f0c686c30698` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:48:59.158122 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb505fc3-d5b0-41a3-ac0b-b6aac28817b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e2686e18e184f0f8d08f0c686c30698)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:48:59.160862 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb505fc3-d5b0-41a3-ac0b-b6aac28817b2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:48:59.420505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a621b703-3c51-4154-9dd0-951c0c8a9d1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '872b8c3a69d14610951c1e81a95d1996'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:59.423693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:48:59.431903 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cb505fc3-d5b0-41a3-ac0b-b6aac28817b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e2686e18e184f0f8d08f0c686c30698'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:59.521782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a621b703-3c51-4154-9dd0-951c0c8a9d1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '872b8c3a69d14610951c1e81a95d1996'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:59.529179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a621b703-3c51-4154-9dd0-951c0c8a9d1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '872b8c3a69d14610951c1e81a95d1996'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:59.541179 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a621b703-3c51-4154-9dd0-951c0c8a9d1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '872b8c3a69d14610951c1e81a95d1996'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:59.541896 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a621b703-3c51-4154-9dd0-951c0c8a9d1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '872b8c3a69d14610951c1e81a95d1996'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:59.547912 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1330/2651] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:59 2022] DELETE /identity/v3/users/872b8c3a69d14610951c1e81a95d1996 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:48:59.602418 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cb505fc3-d5b0-41a3-ac0b-b6aac28817b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e2686e18e184f0f8d08f0c686c30698'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:59.603077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cb505fc3-d5b0-41a3-ac0b-b6aac28817b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e2686e18e184f0f8d08f0c686c30698'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:59.603698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cb505fc3-d5b0-41a3-ac0b-b6aac28817b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e2686e18e184f0f8d08f0c686c30698'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:59.611765 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-cb505fc3-d5b0-41a3-ac0b-b6aac28817b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e2686e18e184f0f8d08f0c686c30698'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:48:59.630681 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1322/2652] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:48:59 2022] DELETE /identity/v3/users/1e2686e18e184f0f8d08f0c686c30698 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:49:00.096165 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621b703-3c51-4154-9dd0-951c0c8a9d1b admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:00.098381 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a621b703-3c51-4154-9dd0-951c0c8a9d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:00.165759 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621b703-3c51-4154-9dd0-951c0c8a9d1b admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:00.223251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4ce77-805a-4fe5-b34b-9b596d957d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:00.224110 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4ce77-805a-4fe5-b34b-9b596d957d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:00.224476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4ce77-805a-4fe5-b34b-9b596d957d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:00.224753 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4ce77-805a-4fe5-b34b-9b596d957d43 admin admin] PATH_INFO: `/v3/projects/8829cb0816b24a1585292916f2876aaf` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:00.226042 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4ce77-805a-4fe5-b34b-9b596d957d43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8829cb0816b24a1585292916f2876aaf)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:00.227936 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4ce77-805a-4fe5-b34b-9b596d957d43 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:00.249501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb505fc3-d5b0-41a3-ac0b-b6aac28817b2 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:00.251013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-cb505fc3-d5b0-41a3-ac0b-b6aac28817b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:00.266807 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb505fc3-d5b0-41a3-ac0b-b6aac28817b2 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:00.272501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef246f4-d2c9-4051-a4e2-c725fd01409d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:00.273765 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef246f4-d2c9-4051-a4e2-c725fd01409d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:00.274055 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef246f4-d2c9-4051-a4e2-c725fd01409d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:00.274338 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef246f4-d2c9-4051-a4e2-c725fd01409d admin admin] PATH_INFO: `/v3/projects/c4ea07a3c2264398a3d6b232ea250fea` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:00.275580 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef246f4-d2c9-4051-a4e2-c725fd01409d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4ea07a3c2264398a3d6b232ea250fea)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:00.277558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef246f4-d2c9-4051-a4e2-c725fd01409d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:00.306445 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:00.339750 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-97c4ce77-805a-4fe5-b34b-9b596d957d43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8829cb0816b24a1585292916f2876aaf', 'request_id': 'req-97c4ce77-805a-4fe5-b34b-9b596d957d43'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:00.344983 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-97c4ce77-805a-4fe5-b34b-9b596d957d43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8829cb0816b24a1585292916f2876aaf', 'request_id': 'req-97c4ce77-805a-4fe5-b34b-9b596d957d43'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:00.351655 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-97c4ce77-805a-4fe5-b34b-9b596d957d43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8829cb0816b24a1585292916f2876aaf', 'request_id': 'req-97c4ce77-805a-4fe5-b34b-9b596d957d43'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:00.425758 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6ef246f4-d2c9-4051-a4e2-c725fd01409d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4ea07a3c2264398a3d6b232ea250fea', 'request_id': 'req-6ef246f4-d2c9-4051-a4e2-c725fd01409d'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:00.432721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6ef246f4-d2c9-4051-a4e2-c725fd01409d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4ea07a3c2264398a3d6b232ea250fea', 'request_id': 'req-6ef246f4-d2c9-4051-a4e2-c725fd01409d'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:00.435386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-97c4ce77-805a-4fe5-b34b-9b596d957d43 admin admin] The token cache is being invalidate because project 8829cb0816b24a1585292916f2876aaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:49:00.435675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-97c4ce77-805a-4fe5-b34b-9b596d957d43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:00.436295 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-97c4ce77-805a-4fe5-b34b-9b596d957d43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:00.437827 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1331/2653] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:49:00 2022] DELETE /identity/v3/projects/8829cb0816b24a1585292916f2876aaf => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:49:00.440084 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6ef246f4-d2c9-4051-a4e2-c725fd01409d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4ea07a3c2264398a3d6b232ea250fea', 'request_id': 'req-6ef246f4-d2c9-4051-a4e2-c725fd01409d'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:00.442006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4ce77-805a-4fe5-b34b-9b596d957d43 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:00.443486 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-97c4ce77-805a-4fe5-b34b-9b596d957d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:00.450233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:00.510017 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4ce77-805a-4fe5-b34b-9b596d957d43 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:00.542924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6ef246f4-d2c9-4051-a4e2-c725fd01409d admin admin] The token cache is being invalidate because project c4ea07a3c2264398a3d6b232ea250fea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:49:00.543228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6ef246f4-d2c9-4051-a4e2-c725fd01409d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:00.543857 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6ef246f4-d2c9-4051-a4e2-c725fd01409d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:00.545616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1323/2654] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:49:00 2022] DELETE /identity/v3/projects/c4ea07a3c2264398a3d6b232ea250fea => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:49:00.550113 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ef083-7c40-4d48-aa9f-08241abe51f5 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:00.550394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ef083-7c40-4d48-aa9f-08241abe51f5 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:00.551017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ef083-7c40-4d48-aa9f-08241abe51f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:00.570665 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa8b43-cb7d-47d6-a63e-ff8109d630eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:00.576135 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-b46ef083-7c40-4d48-aa9f-08241abe51f5 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:00.576251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa8b43-cb7d-47d6-a63e-ff8109d630eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:00.576251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa8b43-cb7d-47d6-a63e-ff8109d630eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:00.576251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa8b43-cb7d-47d6-a63e-ff8109d630eb admin admin] PATH_INFO: `/v3/users/eebce363f4b544cea46e36140653d08b` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:00.576435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b46ef083-7c40-4d48-aa9f-08241abe51f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:00.576501 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa8b43-cb7d-47d6-a63e-ff8109d630eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=eebce363f4b544cea46e36140653d08b)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:00.576501 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa8b43-cb7d-47d6-a63e-ff8109d630eb admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:00.580424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1324/2655] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:49:00.588863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd3114-f001-4bfb-8967-3ec16c7ef246 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:00.589197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd3114-f001-4bfb-8967-3ec16c7ef246 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:00.589478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd3114-f001-4bfb-8967-3ec16c7ef246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:00.609614 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f2dd3114-f001-4bfb-8967-3ec16c7ef246 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:00.610846 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f2dd3114-f001-4bfb-8967-3ec16c7ef246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:00.620285 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1325/2656] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:49:00.624591 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd3114-f001-4bfb-8967-3ec16c7ef246 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:00.634758 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd3114-f001-4bfb-8967-3ec16c7ef246 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:00.639714 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3aec84-e31c-4b31-a447-c7b31d19fbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:00.640407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3aec84-e31c-4b31-a447-c7b31d19fbc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:00.640686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3aec84-e31c-4b31-a447-c7b31d19fbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:00.640967 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3aec84-e31c-4b31-a447-c7b31d19fbc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:00.641707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3aec84-e31c-4b31-a447-c7b31d19fbc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:49:00.642013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3aec84-e31c-4b31-a447-c7b31d19fbc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:00.644254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3aec84-e31c-4b31-a447-c7b31d19fbc4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:00.658164 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1326/2657] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:49:00.670421 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3aec84-e31c-4b31-a447-c7b31d19fbc4 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:00.681646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3aec84-e31c-4b31-a447-c7b31d19fbc4 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:00.687449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5a063-b369-481d-a1c4-28fba0b0dbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:00.688131 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5a063-b369-481d-a1c4-28fba0b0dbba admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:00.688413 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5a063-b369-481d-a1c4-28fba0b0dbba admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:00.688685 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5a063-b369-481d-a1c4-28fba0b0dbba admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:00.689586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5a063-b369-481d-a1c4-28fba0b0dbba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:00.691665 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5a063-b369-481d-a1c4-28fba0b0dbba admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:00.803578 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1327/2658] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:49:00 2022] POST /identity/v3/projects => generated 383 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:49:00.807824 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5a063-b369-481d-a1c4-28fba0b0dbba admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:00.869475 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5a063-b369-481d-a1c4-28fba0b0dbba admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:00.874063 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd292419-c004-4149-b8d3-fdcb9a50e946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:00.874794 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd292419-c004-4149-b8d3-fdcb9a50e946 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:00.875110 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd292419-c004-4149-b8d3-fdcb9a50e946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:00.875389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd292419-c004-4149-b8d3-fdcb9a50e946 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:00.876080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd292419-c004-4149-b8d3-fdcb9a50e946 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:00.878511 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd292419-c004-4149-b8d3-fdcb9a50e946 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:00.902186 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-62aa8b43-cb7d-47d6-a63e-ff8109d630eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eebce363f4b544cea46e36140653d08b'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:00.974222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1328/2659] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:49:00 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:49:00.979395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd292419-c004-4149-b8d3-fdcb9a50e946 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:00.979910 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-62aa8b43-cb7d-47d6-a63e-ff8109d630eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eebce363f4b544cea46e36140653d08b'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:00.989310 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-62aa8b43-cb7d-47d6-a63e-ff8109d630eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eebce363f4b544cea46e36140653d08b'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:00.992870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd292419-c004-4149-b8d3-fdcb9a50e946 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:00.997844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38de24-902d-4042-810f-6fcb8bae35fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:00.998531 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38de24-902d-4042-810f-6fcb8bae35fb admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:00.998813 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38de24-902d-4042-810f-6fcb8bae35fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:00.999117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38de24-902d-4042-810f-6fcb8bae35fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:00.999841 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38de24-902d-4042-810f-6fcb8bae35fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:01.001462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38de24-902d-4042-810f-6fcb8bae35fb admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:01.003749 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-62aa8b43-cb7d-47d6-a63e-ff8109d630eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eebce363f4b544cea46e36140653d08b'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:01.004453 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-62aa8b43-cb7d-47d6-a63e-ff8109d630eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eebce363f4b544cea46e36140653d08b'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:01.011447 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1332/2660] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:49:00 2022] DELETE /identity/v3/users/eebce363f4b544cea46e36140653d08b => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:49:01.015993 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1329/2661] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:49:01.017490 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa8b43-cb7d-47d6-a63e-ff8109d630eb admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:01.019496 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-62aa8b43-cb7d-47d6-a63e-ff8109d630eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:01.054630 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:01.086230 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa8b43-cb7d-47d6-a63e-ff8109d630eb admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:01.090935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879b703-fdd7-498c-acb3-1c9a1454828f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:01.091599 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879b703-fdd7-498c-acb3-1c9a1454828f admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:01.091873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879b703-fdd7-498c-acb3-1c9a1454828f admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:01.092428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879b703-fdd7-498c-acb3-1c9a1454828f admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:01.093031 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d879b703-fdd7-498c-acb3-1c9a1454828f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:01.094603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d879b703-fdd7-498c-acb3-1c9a1454828f admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:01.105025 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1333/2662] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:49:01.109562 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38de24-902d-4042-810f-6fcb8bae35fb admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:01.124424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38de24-902d-4042-810f-6fcb8bae35fb admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:01.126138 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc8217-dc83-47d3-b71e-61d906fd2ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:01.127052 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc8217-dc83-47d3-b71e-61d906fd2ec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:01.127337 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc8217-dc83-47d3-b71e-61d906fd2ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:01.127614 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc8217-dc83-47d3-b71e-61d906fd2ec1 admin admin] PATH_INFO: `/v3/projects/cc0a9524aa014d2d9a8c35b613cb2701/users/4c6c8a43eb7c4f0f8d6446109fce3a86/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:01.143068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc8217-dc83-47d3-b71e-61d906fd2ec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc0a9524aa014d2d9a8c35b613cb2701, user_id=4c6c8a43eb7c4f0f8d6446109fce3a86, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:01.146400 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc8217-dc83-47d3-b71e-61d906fd2ec1 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:01.230863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1330/2663] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:49:01 2022] PUT /identity/v3/projects/cc0a9524aa014d2d9a8c35b613cb2701/users/4c6c8a43eb7c4f0f8d6446109fce3a86/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:49:01.242662 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1810456-15cb-4494-b688-88dcc84f454f None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:01.242986 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1810456-15cb-4494-b688-88dcc84f454f None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:01.243263 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1810456-15cb-4494-b688-88dcc84f454f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:01.274526 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-e1810456-15cb-4494-b688-88dcc84f454f None None] MFA Rules not processed for user `4c6c8a43eb7c4f0f8d6446109fce3a86`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:01.275783 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e1810456-15cb-4494-b688-88dcc84f454f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:01.346177 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1334/2664] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:49:01.357783 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc8217-dc83-47d3-b71e-61d906fd2ec1 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:01.359566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-75fc8217-dc83-47d3-b71e-61d906fd2ec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:01.371101 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:01.402011 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1810456-15cb-4494-b688-88dcc84f454f None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:01.403872 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e1810456-15cb-4494-b688-88dcc84f454f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:01.421608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc8217-dc83-47d3-b71e-61d906fd2ec1 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:01.427470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6232c43-3606-439a-9197-621bbecdb6b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:01.427981 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6232c43-3606-439a-9197-621bbecdb6b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:01.428264 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6232c43-3606-439a-9197-621bbecdb6b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:01.429107 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6232c43-3606-439a-9197-621bbecdb6b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:01.438773 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6232c43-3606-439a-9197-621bbecdb6b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:01.440347 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6232c43-3606-439a-9197-621bbecdb6b7 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:01.450342 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1331/2665] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:49:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:49:01.470050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6232c43-3606-439a-9197-621bbecdb6b7 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:01.473438 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1810456-15cb-4494-b688-88dcc84f454f None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:01.477938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd828f8-5bd8-49b3-b89d-8ab71e296527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:01.478788 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd828f8-5bd8-49b3-b89d-8ab71e296527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:01.479096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd828f8-5bd8-49b3-b89d-8ab71e296527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:01.479366 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd828f8-5bd8-49b3-b89d-8ab71e296527 admin admin] PATH_INFO: `/v3/projects/da8e51cc09d043ffaa407cba35b1a91b` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:01.480557 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd828f8-5bd8-49b3-b89d-8ab71e296527 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da8e51cc09d043ffaa407cba35b1a91b)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:01.481092 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6232c43-3606-439a-9197-621bbecdb6b7 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:01.482455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd828f8-5bd8-49b3-b89d-8ab71e296527 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:01.486500 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079081b-0d23-43e8-b8fc-cbc5f21c89c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:01.487895 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079081b-0d23-43e8-b8fc-cbc5f21c89c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:01.488201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079081b-0d23-43e8-b8fc-cbc5f21c89c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:01.488626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079081b-0d23-43e8-b8fc-cbc5f21c89c4 admin admin] PATH_INFO: `/v3/users/4c6c8a43eb7c4f0f8d6446109fce3a86` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:01.489840 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079081b-0d23-43e8-b8fc-cbc5f21c89c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c6c8a43eb7c4f0f8d6446109fce3a86)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:01.491786 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079081b-0d23-43e8-b8fc-cbc5f21c89c4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:01.590359 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-bcd828f8-5bd8-49b3-b89d-8ab71e296527 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da8e51cc09d043ffaa407cba35b1a91b', 'request_id': 'req-bcd828f8-5bd8-49b3-b89d-8ab71e296527'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:01.596082 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-bcd828f8-5bd8-49b3-b89d-8ab71e296527 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da8e51cc09d043ffaa407cba35b1a91b', 'request_id': 'req-bcd828f8-5bd8-49b3-b89d-8ab71e296527'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:01.597720 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:01.603470 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-bcd828f8-5bd8-49b3-b89d-8ab71e296527 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da8e51cc09d043ffaa407cba35b1a91b', 'request_id': 'req-bcd828f8-5bd8-49b3-b89d-8ab71e296527'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:01.715353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-bcd828f8-5bd8-49b3-b89d-8ab71e296527 admin admin] The token cache is being invalidate because project da8e51cc09d043ffaa407cba35b1a91b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:49:01.715591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-bcd828f8-5bd8-49b3-b89d-8ab71e296527 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:01.716221 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-bcd828f8-5bd8-49b3-b89d-8ab71e296527 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:01.717725 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1335/2666] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:49:01 2022] DELETE /identity/v3/projects/da8e51cc09d043ffaa407cba35b1a91b => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:49:01.724063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bb736-5fa4-4541-a97c-a3785eba5a54 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:01.724348 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bb736-5fa4-4541-a97c-a3785eba5a54 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:01.724619 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bb736-5fa4-4541-a97c-a3785eba5a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:01.725190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6079081b-0d23-43e8-b8fc-cbc5f21c89c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c6c8a43eb7c4f0f8d6446109fce3a86'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:01.743140 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-919bb736-5fa4-4541-a97c-a3785eba5a54 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:01.744340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-919bb736-5fa4-4541-a97c-a3785eba5a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:01.801669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1336/2667] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:49:01.809681 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fc762-87d7-4aa4-a4e0-0349b5856c62 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:01.809969 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fc762-87d7-4aa4-a4e0-0349b5856c62 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:01.810246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fc762-87d7-4aa4-a4e0-0349b5856c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:01.813156 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6079081b-0d23-43e8-b8fc-cbc5f21c89c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c6c8a43eb7c4f0f8d6446109fce3a86'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:01.813853 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6079081b-0d23-43e8-b8fc-cbc5f21c89c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c6c8a43eb7c4f0f8d6446109fce3a86'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:01.814516 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6079081b-0d23-43e8-b8fc-cbc5f21c89c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c6c8a43eb7c4f0f8d6446109fce3a86'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:01.822751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6079081b-0d23-43e8-b8fc-cbc5f21c89c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c6c8a43eb7c4f0f8d6446109fce3a86'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:01.828083 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-1d5fc762-87d7-4aa4-a4e0-0349b5856c62 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:01.829390 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-1d5fc762-87d7-4aa4-a4e0-0349b5856c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:01.837813 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1337/2668] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:49:01.840189 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1332/2669] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:49:01 2022] DELETE /identity/v3/users/4c6c8a43eb7c4f0f8d6446109fce3a86 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:49:01.841925 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5fc762-87d7-4aa4-a4e0-0349b5856c62 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:01.844333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7b2a2-9d0a-41ec-a178-254f4bd94548 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:01.844625 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7b2a2-9d0a-41ec-a178-254f4bd94548 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:01.844899 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7b2a2-9d0a-41ec-a178-254f4bd94548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:01.852076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5fc762-87d7-4aa4-a4e0-0349b5856c62 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:01.864114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3fb7b2a2-9d0a-41ec-a178-254f4bd94548 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:01.865361 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3fb7b2a2-9d0a-41ec-a178-254f4bd94548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:01.909873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7432f55-f222-414d-9440-f5a595fc8afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:01.910541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7432f55-f222-414d-9440-f5a595fc8afa admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:01.910818 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7432f55-f222-414d-9440-f5a595fc8afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:01.911119 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7432f55-f222-414d-9440-f5a595fc8afa admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:01.911968 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7432f55-f222-414d-9440-f5a595fc8afa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:49:01.912280 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7432f55-f222-414d-9440-f5a595fc8afa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:01.913920 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7432f55-f222-414d-9440-f5a595fc8afa admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:01.926336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1338/2670] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 15:49:01.930705 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7432f55-f222-414d-9440-f5a595fc8afa admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:01.936048 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1333/2671] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:49:01.940465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7432f55-f222-414d-9440-f5a595fc8afa admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:01.945418 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb993f1-fcc3-4242-af70-09efd9f95cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:01.945566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7b2a2-9d0a-41ec-a178-254f4bd94548 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:01.946066 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb993f1-fcc3-4242-af70-09efd9f95cbf admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:01.946342 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb993f1-fcc3-4242-af70-09efd9f95cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:01.946611 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb993f1-fcc3-4242-af70-09efd9f95cbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:01.946863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3fb7b2a2-9d0a-41ec-a178-254f4bd94548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:01.947327 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb993f1-fcc3-4242-af70-09efd9f95cbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:01.949354 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb993f1-fcc3-4242-af70-09efd9f95cbf admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:01.976279 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:02.004942 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7b2a2-9d0a-41ec-a178-254f4bd94548 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:02.008585 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be21463-ccdd-43f8-b8bf-b825fa64f4f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:02.009226 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be21463-ccdd-43f8-b8bf-b825fa64f4f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:02.009510 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be21463-ccdd-43f8-b8bf-b825fa64f4f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:02.009781 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be21463-ccdd-43f8-b8bf-b825fa64f4f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:02.019303 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be21463-ccdd-43f8-b8bf-b825fa64f4f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:02.021030 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be21463-ccdd-43f8-b8bf-b825fa64f4f4 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:02.030600 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1334/2672] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:49:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:49:02.084398 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1339/2673] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:49:01 2022] POST /identity/v3/projects => generated 365 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:49:02.093699 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be21463-ccdd-43f8-b8bf-b825fa64f4f4 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:02.104033 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be21463-ccdd-43f8-b8bf-b825fa64f4f4 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:02.160488 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d43fa-07c0-4614-a71b-66d1c8d9bae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:02.161180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d43fa-07c0-4614-a71b-66d1c8d9bae8 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:02.161457 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d43fa-07c0-4614-a71b-66d1c8d9bae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:02.161728 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d43fa-07c0-4614-a71b-66d1c8d9bae8 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:02.162408 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d43fa-07c0-4614-a71b-66d1c8d9bae8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:02.164350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d43fa-07c0-4614-a71b-66d1c8d9bae8 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:02.287070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1335/2674] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:49:02 2022] POST /identity/v3/users => generated 283 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:49:02.293674 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb993f1-fcc3-4242-af70-09efd9f95cbf admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:02.299601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb993f1-fcc3-4242-af70-09efd9f95cbf admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:02.304409 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbeea7d-323a-4942-85ab-24e9b96e60a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:02.305211 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbeea7d-323a-4942-85ab-24e9b96e60a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:02.305499 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbeea7d-323a-4942-85ab-24e9b96e60a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:02.305800 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbeea7d-323a-4942-85ab-24e9b96e60a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:02.306533 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbeea7d-323a-4942-85ab-24e9b96e60a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:02.308164 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbeea7d-323a-4942-85ab-24e9b96e60a3 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:02.319438 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1340/2675] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:49:02.323536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d43fa-07c0-4614-a71b-66d1c8d9bae8 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:02.334597 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d43fa-07c0-4614-a71b-66d1c8d9bae8 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:02.339352 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986547e-c9f7-492b-bea0-a6c10a601d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:02.340024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986547e-c9f7-492b-bea0-a6c10a601d2d admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:02.340307 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986547e-c9f7-492b-bea0-a6c10a601d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:02.340586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986547e-c9f7-492b-bea0-a6c10a601d2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:02.341313 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986547e-c9f7-492b-bea0-a6c10a601d2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:02.342969 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986547e-c9f7-492b-bea0-a6c10a601d2d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:02.353449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1336/2676] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:49:02.357767 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbeea7d-323a-4942-85ab-24e9b96e60a3 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:02.368207 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbeea7d-323a-4942-85ab-24e9b96e60a3 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:02.373749 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd79b84-5ada-446f-a126-c9c25d66fc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:02.374851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd79b84-5ada-446f-a126-c9c25d66fc56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:02.375292 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd79b84-5ada-446f-a126-c9c25d66fc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:02.375578 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd79b84-5ada-446f-a126-c9c25d66fc56 admin admin] PATH_INFO: `/v3/projects/c9e12943e9f943a294cd50279c3aabbc/users/89955a2cdcb745f582e907294bc4c27a/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:02.393060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd79b84-5ada-446f-a126-c9c25d66fc56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9e12943e9f943a294cd50279c3aabbc, user_id=89955a2cdcb745f582e907294bc4c27a, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:02.396462 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd79b84-5ada-446f-a126-c9c25d66fc56 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:02.475956 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986547e-c9f7-492b-bea0-a6c10a601d2d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:02.477741 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-b986547e-c9f7-492b-bea0-a6c10a601d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:02.491420 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986547e-c9f7-492b-bea0-a6c10a601d2d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:02.500178 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5e7cd-9d34-4439-9e50-e77aee0f7923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:02.500178 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e7cd-9d34-4439-9e50-e77aee0f7923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:02.500178 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e7cd-9d34-4439-9e50-e77aee0f7923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:02.500178 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e7cd-9d34-4439-9e50-e77aee0f7923 admin admin] PATH_INFO: `/v3/projects/cc0a9524aa014d2d9a8c35b613cb2701` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:02.500649 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5e7cd-9d34-4439-9e50-e77aee0f7923 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc0a9524aa014d2d9a8c35b613cb2701)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:02.502477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5e7cd-9d34-4439-9e50-e77aee0f7923 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:02.504796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1341/2677] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:49:02 2022] PUT /identity/v3/projects/c9e12943e9f943a294cd50279c3aabbc/users/89955a2cdcb745f582e907294bc4c27a/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:49:02.515866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71471f4-d343-476f-9b75-db6d6ea5870a None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:02.516149 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71471f4-d343-476f-9b75-db6d6ea5870a None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:02.516424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71471f4-d343-476f-9b75-db6d6ea5870a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:02.545445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-b71471f4-d343-476f-9b75-db6d6ea5870a None None] MFA Rules not processed for user `89955a2cdcb745f582e907294bc4c27a`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:02.546669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b71471f4-d343-476f-9b75-db6d6ea5870a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:02.551813 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:02.611216 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1342/2678] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:49:02.615793 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ebf5e7cd-9d34-4439-9e50-e77aee0f7923 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc0a9524aa014d2d9a8c35b613cb2701', 'request_id': 'req-ebf5e7cd-9d34-4439-9e50-e77aee0f7923'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:02.620427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71471f4-d343-476f-9b75-db6d6ea5870a None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:02.621753 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ebf5e7cd-9d34-4439-9e50-e77aee0f7923 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc0a9524aa014d2d9a8c35b613cb2701', 'request_id': 'req-ebf5e7cd-9d34-4439-9e50-e77aee0f7923'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:02.630786 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ebf5e7cd-9d34-4439-9e50-e77aee0f7923 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc0a9524aa014d2d9a8c35b613cb2701', 'request_id': 'req-ebf5e7cd-9d34-4439-9e50-e77aee0f7923'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:02.682267 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71471f4-d343-476f-9b75-db6d6ea5870a None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:02.705738 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ebf5e7cd-9d34-4439-9e50-e77aee0f7923 admin admin] The token cache is being invalidate because project cc0a9524aa014d2d9a8c35b613cb2701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:49:02.706037 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ebf5e7cd-9d34-4439-9e50-e77aee0f7923 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:02.707413 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-ebf5e7cd-9d34-4439-9e50-e77aee0f7923 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:02.709181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1337/2679] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:49:02 2022] DELETE /identity/v3/projects/cc0a9524aa014d2d9a8c35b613cb2701 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:49:02.714509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbf9d4-22a5-409f-984d-6f74e438fc5d None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:02.714797 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbf9d4-22a5-409f-984d-6f74e438fc5d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:02.715107 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbf9d4-22a5-409f-984d-6f74e438fc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:02.733313 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-32fbf9d4-22a5-409f-984d-6f74e438fc5d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:02.734537 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-32fbf9d4-22a5-409f-984d-6f74e438fc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:02.739279 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84463a2b-9116-4364-acbf-1ce74b3303ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:02.739942 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84463a2b-9116-4364-acbf-1ce74b3303ce admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:02.740226 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84463a2b-9116-4364-acbf-1ce74b3303ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:02.740500 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84463a2b-9116-4364-acbf-1ce74b3303ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:02.741180 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84463a2b-9116-4364-acbf-1ce74b3303ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:02.743109 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84463a2b-9116-4364-acbf-1ce74b3303ce admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:02.743363 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1338/2680] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:49:02.751137 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e2170-0173-4b92-9e83-fa79cb06c1e3 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:02.751364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e2170-0173-4b92-9e83-fa79cb06c1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:02.751640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e2170-0173-4b92-9e83-fa79cb06c1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:02.771424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ed4e2170-0173-4b92-9e83-fa79cb06c1e3 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:02.772613 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ed4e2170-0173-4b92-9e83-fa79cb06c1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:02.780993 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1339/2681] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:49:02.785561 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e2170-0173-4b92-9e83-fa79cb06c1e3 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:02.794484 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e2170-0173-4b92-9e83-fa79cb06c1e3 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:02.799293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e8669-5943-42d5-bd43-74218cc999e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:02.800411 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e8669-5943-42d5-bd43-74218cc999e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:02.800737 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e8669-5943-42d5-bd43-74218cc999e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:02.801014 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e8669-5943-42d5-bd43-74218cc999e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:02.801724 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e8669-5943-42d5-bd43-74218cc999e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:49:02.802027 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e8669-5943-42d5-bd43-74218cc999e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:02.803645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e8669-5943-42d5-bd43-74218cc999e8 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:02.814906 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1340/2682] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 15:49:02.820059 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e8669-5943-42d5-bd43-74218cc999e8 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:02.824992 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:02.830224 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e8669-5943-42d5-bd43-74218cc999e8 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:02.832707 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1343/2683] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:49:02 2022] POST /identity/v3/projects => generated 367 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:49:02.836670 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84463a2b-9116-4364-acbf-1ce74b3303ce admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:02.838478 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-84463a2b-9116-4364-acbf-1ce74b3303ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:02.882293 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:02.893638 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd3d65-b3e8-4ca1-8fae-01b3dd064acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:02.894309 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd3d65-b3e8-4ca1-8fae-01b3dd064acc admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:02.894822 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd3d65-b3e8-4ca1-8fae-01b3dd064acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:02.895132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd3d65-b3e8-4ca1-8fae-01b3dd064acc admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:02.895826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd3d65-b3e8-4ca1-8fae-01b3dd064acc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:02.898150 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd3d65-b3e8-4ca1-8fae-01b3dd064acc admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:02.903702 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84463a2b-9116-4364-acbf-1ce74b3303ce admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:02.909664 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90444eb-4018-400e-a7b3-7defdb42f5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:02.910760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90444eb-4018-400e-a7b3-7defdb42f5e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:02.910760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90444eb-4018-400e-a7b3-7defdb42f5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:02.910975 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90444eb-4018-400e-a7b3-7defdb42f5e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:02.911663 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90444eb-4018-400e-a7b3-7defdb42f5e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:02.913613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90444eb-4018-400e-a7b3-7defdb42f5e6 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:03.010732 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1341/2684] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:49:02 2022] POST /identity/v3/projects => generated 377 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:49:03.011667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd3d65-b3e8-4ca1-8fae-01b3dd064acc admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:03.022787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1344/2685] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:49:02 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:49:03.023900 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90444eb-4018-400e-a7b3-7defdb42f5e6 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:03.033612 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90444eb-4018-400e-a7b3-7defdb42f5e6 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:03.076439 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd3d65-b3e8-4ca1-8fae-01b3dd064acc admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:03.081455 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6df067-0321-448b-8ed6-cf7447efdb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:03.082123 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6df067-0321-448b-8ed6-cf7447efdb15 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:03.082404 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6df067-0321-448b-8ed6-cf7447efdb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:03.082682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6df067-0321-448b-8ed6-cf7447efdb15 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:03.083413 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6df067-0321-448b-8ed6-cf7447efdb15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:03.085323 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6df067-0321-448b-8ed6-cf7447efdb15 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:03.091078 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb15067-b715-42c6-a851-6fbc2fae10db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:03.091746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb15067-b715-42c6-a851-6fbc2fae10db admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:03.092063 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb15067-b715-42c6-a851-6fbc2fae10db admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:03.092340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb15067-b715-42c6-a851-6fbc2fae10db admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:03.093070 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb15067-b715-42c6-a851-6fbc2fae10db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:03.094740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb15067-b715-42c6-a851-6fbc2fae10db admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:03.105889 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1345/2686] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:49:03.110034 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb15067-b715-42c6-a851-6fbc2fae10db admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:03.120515 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb15067-b715-42c6-a851-6fbc2fae10db admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:03.125104 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286826d-ddaa-4fa6-bd04-91a439594fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:03.125967 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286826d-ddaa-4fa6-bd04-91a439594fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:03.126240 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286826d-ddaa-4fa6-bd04-91a439594fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:03.126515 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286826d-ddaa-4fa6-bd04-91a439594fe7 admin admin] PATH_INFO: `/v3/projects/05bef3c10d51429f812b8a4e3d94fe01/users/413d10088d004e03b381ed3b5c24c91d/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:03.140818 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286826d-ddaa-4fa6-bd04-91a439594fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05bef3c10d51429f812b8a4e3d94fe01, user_id=413d10088d004e03b381ed3b5c24c91d, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:03.143577 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286826d-ddaa-4fa6-bd04-91a439594fe7 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:03.179949 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1342/2687] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:49:03 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:49:03.184051 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6df067-0321-448b-8ed6-cf7447efdb15 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:03.193722 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6df067-0321-448b-8ed6-cf7447efdb15 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:03.198421 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47426a-5beb-4860-a3e5-8a1d9f31435c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:03.199101 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47426a-5beb-4860-a3e5-8a1d9f31435c admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:03.199390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47426a-5beb-4860-a3e5-8a1d9f31435c admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:03.199668 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47426a-5beb-4860-a3e5-8a1d9f31435c admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:03.200439 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47426a-5beb-4860-a3e5-8a1d9f31435c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:03.201990 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47426a-5beb-4860-a3e5-8a1d9f31435c admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:03.211971 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1343/2688] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:49:03.215688 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47426a-5beb-4860-a3e5-8a1d9f31435c admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:03.225883 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47426a-5beb-4860-a3e5-8a1d9f31435c admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:03.230926 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1346/2689] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:49:03 2022] PUT /identity/v3/projects/05bef3c10d51429f812b8a4e3d94fe01/users/413d10088d004e03b381ed3b5c24c91d/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:49:03.234684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286826d-ddaa-4fa6-bd04-91a439594fe7 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:03.284011 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611a351-67f0-4e47-bf36-45a6cbb0cc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:03.285275 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611a351-67f0-4e47-bf36-45a6cbb0cc95 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:03.285571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611a351-67f0-4e47-bf36-45a6cbb0cc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:03.285892 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611a351-67f0-4e47-bf36-45a6cbb0cc95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:03.286672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1611a351-67f0-4e47-bf36-45a6cbb0cc95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:03.288320 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1611a351-67f0-4e47-bf36-45a6cbb0cc95 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:03.297349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286826d-ddaa-4fa6-bd04-91a439594fe7 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:03.298458 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1344/2690] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:49:03.303080 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611a351-67f0-4e47-bf36-45a6cbb0cc95 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:03.305803 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb23e83-ee1c-4431-a943-87f983ff92d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:03.306475 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb23e83-ee1c-4431-a943-87f983ff92d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:03.306760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb23e83-ee1c-4431-a943-87f983ff92d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:03.307162 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb23e83-ee1c-4431-a943-87f983ff92d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:03.307830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb23e83-ee1c-4431-a943-87f983ff92d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:03.308243 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:03.309433 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb23e83-ee1c-4431-a943-87f983ff92d9 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:03.313204 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611a351-67f0-4e47-bf36-45a6cbb0cc95 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:03.318292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700496da-af29-45df-b4be-d78796ce9ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:03.323800 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700496da-af29-45df-b4be-d78796ce9ff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:03.323800 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700496da-af29-45df-b4be-d78796ce9ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:03.323800 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700496da-af29-45df-b4be-d78796ce9ff7 admin admin] PATH_INFO: `/v3/projects/748ea5556e184036bad94204b871c850/users/20d85a3033f44696a62dea99859227a5/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:03.324026 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1347/2691] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:49:03.325112 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb23e83-ee1c-4431-a943-87f983ff92d9 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:03.335193 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb23e83-ee1c-4431-a943-87f983ff92d9 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:03.335911 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700496da-af29-45df-b4be-d78796ce9ff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=748ea5556e184036bad94204b871c850, user_id=20d85a3033f44696a62dea99859227a5, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:03.338609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700496da-af29-45df-b4be-d78796ce9ff7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:03.339968 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b13ca5-461d-4d7d-8240-2a4499a65090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:03.340838 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b13ca5-461d-4d7d-8240-2a4499a65090 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:03.341121 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b13ca5-461d-4d7d-8240-2a4499a65090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:03.341694 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b13ca5-461d-4d7d-8240-2a4499a65090 admin admin] PATH_INFO: `/v3/projects/05bef3c10d51429f812b8a4e3d94fe01/users/413d10088d004e03b381ed3b5c24c91d/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:03.343449 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b13ca5-461d-4d7d-8240-2a4499a65090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05bef3c10d51429f812b8a4e3d94fe01, user_id=413d10088d004e03b381ed3b5c24c91d, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:03.346052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b13ca5-461d-4d7d-8240-2a4499a65090 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:03.356936 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1348/2692] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:49:03 2022] PUT /identity/v3/projects/05bef3c10d51429f812b8a4e3d94fe01/users/413d10088d004e03b381ed3b5c24c91d/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:49:03.368349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a46520-58ad-438f-afe0-4fa4b4f34efb None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:03.368636 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a46520-58ad-438f-afe0-4fa4b4f34efb None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:03.368914 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a46520-58ad-438f-afe0-4fa4b4f34efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:03.394999 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-a1a46520-58ad-438f-afe0-4fa4b4f34efb None None] MFA Rules not processed for user `413d10088d004e03b381ed3b5c24c91d`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:03.396216 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a1a46520-58ad-438f-afe0-4fa4b4f34efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:03.402095 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1345/2693] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:49:03 2022] PUT /identity/v3/projects/748ea5556e184036bad94204b871c850/users/20d85a3033f44696a62dea99859227a5/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:49:03.409642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:03.413503 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedbf75-d044-4100-9cb0-7b65f5ba10e4 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:03.413784 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedbf75-d044-4100-9cb0-7b65f5ba10e4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:03.414058 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beedbf75-d044-4100-9cb0-7b65f5ba10e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:03.447482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-beedbf75-d044-4100-9cb0-7b65f5ba10e4 None None] MFA Rules not processed for user `20d85a3033f44696a62dea99859227a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:03.448848 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-beedbf75-d044-4100-9cb0-7b65f5ba10e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:03.474023 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1349/2694] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:03 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:49:03.491978 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a46520-58ad-438f-afe0-4fa4b4f34efb None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:03.493409 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a1a46520-58ad-438f-afe0-4fa4b4f34efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:03.520480 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1346/2695] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:49:03.532286 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedbf75-d044-4100-9cb0-7b65f5ba10e4 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:03.533651 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-beedbf75-d044-4100-9cb0-7b65f5ba10e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:03.553080 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a46520-58ad-438f-afe0-4fa4b4f34efb None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:03.556712 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d50aa-bb24-413b-b6e7-42bd84e369f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:03.557353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d50aa-bb24-413b-b6e7-42bd84e369f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:03.557629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d50aa-bb24-413b-b6e7-42bd84e369f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:03.557899 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d50aa-bb24-413b-b6e7-42bd84e369f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:03.567259 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d50aa-bb24-413b-b6e7-42bd84e369f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:03.568707 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d50aa-bb24-413b-b6e7-42bd84e369f0 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:03.578312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1350/2696] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:49:03 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:49:03.592071 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beedbf75-d044-4100-9cb0-7b65f5ba10e4 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:03.595768 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c936c28-5597-43ff-aece-e41ada6c4627 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d689162291f4f4b97e886ea20d3a8b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:03.596420 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c936c28-5597-43ff-aece-e41ada6c4627 service glance] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:03.597286 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c936c28-5597-43ff-aece-e41ada6c4627 service glance] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:03.597639 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c936c28-5597-43ff-aece-e41ada6c4627 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:03.606124 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c936c28-5597-43ff-aece-e41ada6c4627 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:03.607672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c936c28-5597-43ff-aece-e41ada6c4627 service glance] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:03.616646 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1347/2697] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:49:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:49:03.782470 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d50aa-bb24-413b-b6e7-42bd84e369f0 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:03.792107 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d50aa-bb24-413b-b6e7-42bd84e369f0 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:03.795871 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a68b6-e029-4ed0-a813-29257fa35658 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:03.796506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a68b6-e029-4ed0-a813-29257fa35658 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:03.796833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a68b6-e029-4ed0-a813-29257fa35658 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:03.797150 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a68b6-e029-4ed0-a813-29257fa35658 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:03.798951 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5e6a68b6-e029-4ed0-a813-29257fa35658 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:03.859607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a68b6-e029-4ed0-a813-29257fa35658 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:03.861041 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a68b6-e029-4ed0-a813-29257fa35658 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:03.870152 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1351/2698] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:49:03 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:49:08.892717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c936c28-5597-43ff-aece-e41ada6c4627 service glance] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:08.894717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5c936c28-5597-43ff-aece-e41ada6c4627 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:08.973226 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:08.989865 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c936c28-5597-43ff-aece-e41ada6c4627 service glance] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:08.994934 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee179e-45cd-4c8e-8210-f8967a4e2f4c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a30cee766814c8685578819261705ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:08.995610 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee179e-45cd-4c8e-8210-f8967a4e2f4c service placement] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:08.995891 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee179e-45cd-4c8e-8210-f8967a4e2f4c service placement] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:08.996186 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee179e-45cd-4c8e-8210-f8967a4e2f4c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:08.997925 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-05ee179e-45cd-4c8e-8210-f8967a4e2f4c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:09.010069 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee179e-45cd-4c8e-8210-f8967a4e2f4c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:09.012056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee179e-45cd-4c8e-8210-f8967a4e2f4c service placement] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:09.022581 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1348/2699] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:49:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:49:10.071960 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a68b6-e029-4ed0-a813-29257fa35658 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:10.123905 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=100 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:10.136732 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a68b6-e029-4ed0-a813-29257fa35658 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:10.141272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2e24f-7044-4e9b-972f-d732789f7edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:10.142129 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2e24f-7044-4e9b-972f-d732789f7edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:10.142407 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2e24f-7044-4e9b-972f-d732789f7edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:10.146864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2e24f-7044-4e9b-972f-d732789f7edb admin admin] PATH_INFO: `/v3/users/89955a2cdcb745f582e907294bc4c27a` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:10.146864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2e24f-7044-4e9b-972f-d732789f7edb admin admin] RBAC: Authorizing `identity:delete_user(user_id=89955a2cdcb745f582e907294bc4c27a)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:10.146864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2e24f-7044-4e9b-972f-d732789f7edb admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:10.368473 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-50b2e24f-7044-4e9b-972f-d732789f7edb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89955a2cdcb745f582e907294bc4c27a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:10.443363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-50b2e24f-7044-4e9b-972f-d732789f7edb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89955a2cdcb745f582e907294bc4c27a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:10.451915 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-50b2e24f-7044-4e9b-972f-d732789f7edb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89955a2cdcb745f582e907294bc4c27a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:10.463792 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-50b2e24f-7044-4e9b-972f-d732789f7edb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89955a2cdcb745f582e907294bc4c27a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:10.464494 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-50b2e24f-7044-4e9b-972f-d732789f7edb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89955a2cdcb745f582e907294bc4c27a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:10.470456 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1352/2700] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:49:10 2022] DELETE /identity/v3/users/89955a2cdcb745f582e907294bc4c27a => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:49:10.475105 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14090770-4f91-4834-86ad-90ba675e043e None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:10.475576 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14090770-4f91-4834-86ad-90ba675e043e None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:10.475661 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14090770-4f91-4834-86ad-90ba675e043e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:10.494427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-14090770-4f91-4834-86ad-90ba675e043e None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:10.495691 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-14090770-4f91-4834-86ad-90ba675e043e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:10.559862 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1349/2701] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:49:10.570099 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2e24f-7044-4e9b-972f-d732789f7edb admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:10.571640 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-50b2e24f-7044-4e9b-972f-d732789f7edb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:10.629995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2e24f-7044-4e9b-972f-d732789f7edb admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:10.634363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c20a25-39b5-4b0b-a934-95b6c86541ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:10.635045 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c20a25-39b5-4b0b-a934-95b6c86541ee service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:10.635424 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c20a25-39b5-4b0b-a934-95b6c86541ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:10.635593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c20a25-39b5-4b0b-a934-95b6c86541ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:10.645645 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c20a25-39b5-4b0b-a934-95b6c86541ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:10.647095 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c20a25-39b5-4b0b-a934-95b6c86541ee service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:10.657590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1353/2702] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:49:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:49:11.078675 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14090770-4f91-4834-86ad-90ba675e043e None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:11.080229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-14090770-4f91-4834-86ad-90ba675e043e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:11.094752 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14090770-4f91-4834-86ad-90ba675e043e None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:11.100362 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427d76c-913e-4d72-8245-615dddafd904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:11.101267 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427d76c-913e-4d72-8245-615dddafd904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:11.101658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427d76c-913e-4d72-8245-615dddafd904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:11.101800 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427d76c-913e-4d72-8245-615dddafd904 admin admin] PATH_INFO: `/v3/projects/c9e12943e9f943a294cd50279c3aabbc` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:11.103069 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427d76c-913e-4d72-8245-615dddafd904 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9e12943e9f943a294cd50279c3aabbc)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:11.105009 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427d76c-913e-4d72-8245-615dddafd904 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:11.226719 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f427d76c-913e-4d72-8245-615dddafd904 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9e12943e9f943a294cd50279c3aabbc', 'request_id': 'req-f427d76c-913e-4d72-8245-615dddafd904'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:11.232052 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f427d76c-913e-4d72-8245-615dddafd904 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9e12943e9f943a294cd50279c3aabbc', 'request_id': 'req-f427d76c-913e-4d72-8245-615dddafd904'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:11.239205 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f427d76c-913e-4d72-8245-615dddafd904 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9e12943e9f943a294cd50279c3aabbc', 'request_id': 'req-f427d76c-913e-4d72-8245-615dddafd904'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:11.350401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f427d76c-913e-4d72-8245-615dddafd904 admin admin] The token cache is being invalidate because project c9e12943e9f943a294cd50279c3aabbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:49:11.350688 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f427d76c-913e-4d72-8245-615dddafd904 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:11.351350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-f427d76c-913e-4d72-8245-615dddafd904 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:11.353126 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1350/2703] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:49:11 2022] DELETE /identity/v3/projects/c9e12943e9f943a294cd50279c3aabbc => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:49:11.357098 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c20a25-39b5-4b0b-a934-95b6c86541ee service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:11.368760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c20a25-39b5-4b0b-a934-95b6c86541ee service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:11.427855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57393b3b-dca8-46a4-96d2-3673a1614c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:11.428734 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57393b3b-dca8-46a4-96d2-3673a1614c01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:11.429010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57393b3b-dca8-46a4-96d2-3673a1614c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:11.429283 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57393b3b-dca8-46a4-96d2-3673a1614c01 admin admin] PATH_INFO: `/v3/users/413d10088d004e03b381ed3b5c24c91d` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:11.430568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57393b3b-dca8-46a4-96d2-3673a1614c01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=413d10088d004e03b381ed3b5c24c91d)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:11.432548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57393b3b-dca8-46a4-96d2-3673a1614c01 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:11.446738 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:11.620479 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-57393b3b-dca8-46a4-96d2-3673a1614c01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '413d10088d004e03b381ed3b5c24c91d'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:11.685608 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-57393b3b-dca8-46a4-96d2-3673a1614c01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '413d10088d004e03b381ed3b5c24c91d'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:11.694076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-57393b3b-dca8-46a4-96d2-3673a1614c01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '413d10088d004e03b381ed3b5c24c91d'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:11.709430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-57393b3b-dca8-46a4-96d2-3673a1614c01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '413d10088d004e03b381ed3b5c24c91d'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:11.710234 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-57393b3b-dca8-46a4-96d2-3673a1614c01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '413d10088d004e03b381ed3b5c24c91d'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:11.716570 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1354/2704] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:49:11 2022] DELETE /identity/v3/users/413d10088d004e03b381ed3b5c24c91d => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:49:12.141046 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427d76c-913e-4d72-8245-615dddafd904 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:12.143551 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f427d76c-913e-4d72-8245-615dddafd904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:12.158910 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427d76c-913e-4d72-8245-615dddafd904 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:12.221583 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf939538-9a46-4488-829f-9bdceaf2136e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:12.222609 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf939538-9a46-4488-829f-9bdceaf2136e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:12.222914 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf939538-9a46-4488-829f-9bdceaf2136e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:12.223203 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf939538-9a46-4488-829f-9bdceaf2136e admin admin] PATH_INFO: `/v3/projects/05bef3c10d51429f812b8a4e3d94fe01` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:12.224650 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf939538-9a46-4488-829f-9bdceaf2136e admin admin] RBAC: Authorizing `identity:delete_project(project_id=05bef3c10d51429f812b8a4e3d94fe01)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:12.226543 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf939538-9a46-4488-829f-9bdceaf2136e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:12.233496 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:12.403242 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-bf939538-9a46-4488-829f-9bdceaf2136e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05bef3c10d51429f812b8a4e3d94fe01', 'request_id': 'req-bf939538-9a46-4488-829f-9bdceaf2136e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:12.409465 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-bf939538-9a46-4488-829f-9bdceaf2136e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05bef3c10d51429f812b8a4e3d94fe01', 'request_id': 'req-bf939538-9a46-4488-829f-9bdceaf2136e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:12.415982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-bf939538-9a46-4488-829f-9bdceaf2136e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05bef3c10d51429f812b8a4e3d94fe01', 'request_id': 'req-bf939538-9a46-4488-829f-9bdceaf2136e'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:12.493199 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-bf939538-9a46-4488-829f-9bdceaf2136e admin admin] The token cache is being invalidate because project 05bef3c10d51429f812b8a4e3d94fe01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:49:12.493492 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-bf939538-9a46-4488-829f-9bdceaf2136e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:12.494114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-bf939538-9a46-4488-829f-9bdceaf2136e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:12.495750 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1351/2705] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:49:12 2022] DELETE /identity/v3/projects/05bef3c10d51429f812b8a4e3d94fe01 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:49:12.500602 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01675e50-397c-4385-b123-21ff3c3ad3bc None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:12.501031 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01675e50-397c-4385-b123-21ff3c3ad3bc None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:12.501187 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01675e50-397c-4385-b123-21ff3c3ad3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:12.520314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-01675e50-397c-4385-b123-21ff3c3ad3bc None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:12.521552 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-01675e50-397c-4385-b123-21ff3c3ad3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:12.582222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1355/2706] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:49:12.590162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2316d-167c-47b9-8fa4-00c85d97343d None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:12.590451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2316d-167c-47b9-8fa4-00c85d97343d None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:12.590727 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2316d-167c-47b9-8fa4-00c85d97343d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:12.609215 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-cbb2316d-167c-47b9-8fa4-00c85d97343d None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:12.610498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-cbb2316d-167c-47b9-8fa4-00c85d97343d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:12.620234 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1352/2707] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:49:12.624687 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01675e50-397c-4385-b123-21ff3c3ad3bc None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:12.635607 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01675e50-397c-4385-b123-21ff3c3ad3bc None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:12.640190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa270e65-0c87-4749-a2be-a1acae5039e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:12.640837 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa270e65-0c87-4749-a2be-a1acae5039e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:12.641120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa270e65-0c87-4749-a2be-a1acae5039e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:12.641410 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa270e65-0c87-4749-a2be-a1acae5039e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:12.642117 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa270e65-0c87-4749-a2be-a1acae5039e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:49:12.642420 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa270e65-0c87-4749-a2be-a1acae5039e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:12.644038 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa270e65-0c87-4749-a2be-a1acae5039e2 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:12.655719 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1356/2708] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:49:12.659872 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2316d-167c-47b9-8fa4-00c85d97343d None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:12.672288 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2316d-167c-47b9-8fa4-00c85d97343d None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:12.677764 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e616e-91b7-4de7-b57e-7d1a9f5bfea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:12.678428 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e616e-91b7-4de7-b57e-7d1a9f5bfea6 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:12.678710 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e616e-91b7-4de7-b57e-7d1a9f5bfea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:12.679018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e616e-91b7-4de7-b57e-7d1a9f5bfea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:12.679720 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e616e-91b7-4de7-b57e-7d1a9f5bfea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:12.681623 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e616e-91b7-4de7-b57e-7d1a9f5bfea6 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:12.772214 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1353/2709] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:49:12 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:49:12.776174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa270e65-0c87-4749-a2be-a1acae5039e2 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:12.787275 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa270e65-0c87-4749-a2be-a1acae5039e2 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:12.843580 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3222f-c292-40a9-98e3-4737905a5169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:12.844307 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3222f-c292-40a9-98e3-4737905a5169 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:12.844591 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3222f-c292-40a9-98e3-4737905a5169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:12.844914 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3222f-c292-40a9-98e3-4737905a5169 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:12.845628 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3222f-c292-40a9-98e3-4737905a5169 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:12.847741 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3222f-c292-40a9-98e3-4737905a5169 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:12.953471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1357/2710] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:49:12 2022] POST /identity/v3/users => generated 284 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:49:12.954963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e616e-91b7-4de7-b57e-7d1a9f5bfea6 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:12.966197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e616e-91b7-4de7-b57e-7d1a9f5bfea6 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:12.970882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb6649-5912-45df-a823-049bcc0b5923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:12.971571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb6649-5912-45df-a823-049bcc0b5923 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:12.971852 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb6649-5912-45df-a823-049bcc0b5923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:12.972125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb6649-5912-45df-a823-049bcc0b5923 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:12.972846 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb6649-5912-45df-a823-049bcc0b5923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:12.975097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb6649-5912-45df-a823-049bcc0b5923 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:12.985705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1354/2711] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:49:12.989900 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3222f-c292-40a9-98e3-4737905a5169 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:13.000960 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3222f-c292-40a9-98e3-4737905a5169 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:13.005690 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6947ac7-b700-4b26-8164-d5a885ccf745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:13.006357 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6947ac7-b700-4b26-8164-d5a885ccf745 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:13.006640 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6947ac7-b700-4b26-8164-d5a885ccf745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:13.006962 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6947ac7-b700-4b26-8164-d5a885ccf745 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:13.007723 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6947ac7-b700-4b26-8164-d5a885ccf745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:13.009347 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6947ac7-b700-4b26-8164-d5a885ccf745 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:13.011752 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:13.020990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1358/2712] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:49:13.028146 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb6649-5912-45df-a823-049bcc0b5923 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:13.039068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb6649-5912-45df-a823-049bcc0b5923 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:13.044116 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3c84f-bfd8-4580-b12e-194b6a61bf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:13.045061 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3c84f-bfd8-4580-b12e-194b6a61bf57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:13.045355 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3c84f-bfd8-4580-b12e-194b6a61bf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:13.045642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3c84f-bfd8-4580-b12e-194b6a61bf57 admin admin] PATH_INFO: `/v3/projects/5c0bd5c1dd8b4d88be1ded374b9bceef/users/2139919b73cf497f855338b8f90fcfca/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:13.061391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3c84f-bfd8-4580-b12e-194b6a61bf57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c0bd5c1dd8b4d88be1ded374b9bceef, user_id=2139919b73cf497f855338b8f90fcfca, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:13.064082 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3c84f-bfd8-4580-b12e-194b6a61bf57 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:13.133442 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1355/2713] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:49:13 2022] PUT /identity/v3/projects/5c0bd5c1dd8b4d88be1ded374b9bceef/users/2139919b73cf497f855338b8f90fcfca/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:49:13.144478 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a979bb-3ba4-43f6-8939-1861fd85704c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:13.144786 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a979bb-3ba4-43f6-8939-1861fd85704c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:13.145555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a979bb-3ba4-43f6-8939-1861fd85704c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:13.172427 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-61a979bb-3ba4-43f6-8939-1861fd85704c None None] MFA Rules not processed for user `2139919b73cf497f855338b8f90fcfca`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:13.173585 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-61a979bb-3ba4-43f6-8939-1861fd85704c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:13.240787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1359/2714] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:13 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:49:13.252638 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3c84f-bfd8-4580-b12e-194b6a61bf57 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:13.254916 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-aad3c84f-bfd8-4580-b12e-194b6a61bf57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:13.312623 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3c84f-bfd8-4580-b12e-194b6a61bf57 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:13.316280 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c76aa-3302-4312-9dda-6c3c8a24aa72 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d689162291f4f4b97e886ea20d3a8b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:13.317451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c76aa-3302-4312-9dda-6c3c8a24aa72 service glance] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:13.317747 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c76aa-3302-4312-9dda-6c3c8a24aa72 service glance] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:13.318088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c76aa-3302-4312-9dda-6c3c8a24aa72 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:13.326374 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c76aa-3302-4312-9dda-6c3c8a24aa72 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:13.327828 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c76aa-3302-4312-9dda-6c3c8a24aa72 service glance] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:13.337821 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1356/2715] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:49:13 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:49:21.738303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a979bb-3ba4-43f6-8939-1861fd85704c None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:21.739814 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-61a979bb-3ba4-43f6-8939-1861fd85704c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:21.809934 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a979bb-3ba4-43f6-8939-1861fd85704c None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:21.814483 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b6883-be1b-40c9-ae31-415d1c34713d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:21.815389 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b6883-be1b-40c9-ae31-415d1c34713d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:21.815671 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b6883-be1b-40c9-ae31-415d1c34713d admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:21.816014 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b6883-be1b-40c9-ae31-415d1c34713d admin admin] PATH_INFO: `/v3/users/20d85a3033f44696a62dea99859227a5` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:21.817365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b6883-be1b-40c9-ae31-415d1c34713d admin admin] RBAC: Authorizing `identity:delete_user(user_id=20d85a3033f44696a62dea99859227a5)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:21.820886 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b6883-be1b-40c9-ae31-415d1c34713d admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:21.998613 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-fb7b6883-be1b-40c9-ae31-415d1c34713d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20d85a3033f44696a62dea99859227a5'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:22.075654 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-fb7b6883-be1b-40c9-ae31-415d1c34713d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20d85a3033f44696a62dea99859227a5'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:22.083552 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-fb7b6883-be1b-40c9-ae31-415d1c34713d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20d85a3033f44696a62dea99859227a5'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:22.098087 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-fb7b6883-be1b-40c9-ae31-415d1c34713d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20d85a3033f44696a62dea99859227a5'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:22.098828 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-fb7b6883-be1b-40c9-ae31-415d1c34713d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20d85a3033f44696a62dea99859227a5'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:22.105278 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1360/2716] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:49:21 2022] DELETE /identity/v3/users/20d85a3033f44696a62dea99859227a5 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:49:22.109926 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b6b87-e76f-41f0-834d-d4737fdfd788 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:22.110398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b6b87-e76f-41f0-834d-d4737fdfd788 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:22.110598 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b6b87-e76f-41f0-834d-d4737fdfd788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:22.128499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-f42b6b87-e76f-41f0-834d-d4737fdfd788 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:22.129709 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f42b6b87-e76f-41f0-834d-d4737fdfd788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:22.165793 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:22.203774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1357/2717] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:49:22.214130 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b6883-be1b-40c9-ae31-415d1c34713d admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:22.215547 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-fb7b6883-be1b-40c9-ae31-415d1c34713d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:22.226523 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:22.275080 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b6883-be1b-40c9-ae31-415d1c34713d admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:22.278737 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943947c8-d7a1-4920-b59c-ac52cc348b30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:22.279711 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943947c8-d7a1-4920-b59c-ac52cc348b30 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:22.279984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943947c8-d7a1-4920-b59c-ac52cc348b30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:22.280256 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943947c8-d7a1-4920-b59c-ac52cc348b30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:22.290044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943947c8-d7a1-4920-b59c-ac52cc348b30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:22.291503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943947c8-d7a1-4920-b59c-ac52cc348b30 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:22.301397 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1361/2718] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:49:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:49:22.735046 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b6b87-e76f-41f0-834d-d4737fdfd788 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:22.736621 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-f42b6b87-e76f-41f0-834d-d4737fdfd788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:22.750980 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b6b87-e76f-41f0-834d-d4737fdfd788 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:22.755800 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03310478-730d-420f-a1ac-8bcebc725ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:22.757652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03310478-730d-420f-a1ac-8bcebc725ab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:22.757932 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03310478-730d-420f-a1ac-8bcebc725ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:22.758207 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03310478-730d-420f-a1ac-8bcebc725ab4 admin admin] PATH_INFO: `/v3/projects/748ea5556e184036bad94204b871c850` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:22.760377 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03310478-730d-420f-a1ac-8bcebc725ab4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=748ea5556e184036bad94204b871c850)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:22.762284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03310478-730d-420f-a1ac-8bcebc725ab4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:22.860629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-03310478-730d-420f-a1ac-8bcebc725ab4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '748ea5556e184036bad94204b871c850', 'request_id': 'req-03310478-730d-420f-a1ac-8bcebc725ab4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:22.866495 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-03310478-730d-420f-a1ac-8bcebc725ab4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '748ea5556e184036bad94204b871c850', 'request_id': 'req-03310478-730d-420f-a1ac-8bcebc725ab4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:22.873805 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-03310478-730d-420f-a1ac-8bcebc725ab4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '748ea5556e184036bad94204b871c850', 'request_id': 'req-03310478-730d-420f-a1ac-8bcebc725ab4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:22.954975 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-03310478-730d-420f-a1ac-8bcebc725ab4 admin admin] The token cache is being invalidate because project 748ea5556e184036bad94204b871c850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:49:22.955265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-03310478-730d-420f-a1ac-8bcebc725ab4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:22.955897 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-03310478-730d-420f-a1ac-8bcebc725ab4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:22.957473 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1358/2719] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:49:22 2022] DELETE /identity/v3/projects/748ea5556e184036bad94204b871c850 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:49:22.963192 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b390e-ccfe-4399-bff0-41209cf04693 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:22.968088 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b390e-ccfe-4399-bff0-41209cf04693 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:22.968088 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b390e-ccfe-4399-bff0-41209cf04693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:22.981822 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-8e2b390e-ccfe-4399-bff0-41209cf04693 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:22.983087 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-8e2b390e-ccfe-4399-bff0-41209cf04693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:23.044194 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1362/2720] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:49:23.052837 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8f0bb-0069-43df-a74d-cf11109fd06a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:23.053774 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8f0bb-0069-43df-a74d-cf11109fd06a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:23.054075 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8f0bb-0069-43df-a74d-cf11109fd06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:23.074455 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-69c8f0bb-0069-43df-a74d-cf11109fd06a None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:23.075689 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-69c8f0bb-0069-43df-a74d-cf11109fd06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:23.083990 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1359/2721] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:49:23.088833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b390e-ccfe-4399-bff0-41209cf04693 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:23.098565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b390e-ccfe-4399-bff0-41209cf04693 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:23.104027 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8071eb4-0b72-4493-bb66-dd3bb97d8f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:23.104689 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8071eb4-0b72-4493-bb66-dd3bb97d8f59 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:23.104976 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8071eb4-0b72-4493-bb66-dd3bb97d8f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:23.105254 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8071eb4-0b72-4493-bb66-dd3bb97d8f59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:23.105982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8071eb4-0b72-4493-bb66-dd3bb97d8f59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:49:23.106289 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8071eb4-0b72-4493-bb66-dd3bb97d8f59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:23.107919 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8071eb4-0b72-4493-bb66-dd3bb97d8f59 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:23.120214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1363/2722] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:49:23.124525 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8f0bb-0069-43df-a74d-cf11109fd06a None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:23.134726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8f0bb-0069-43df-a74d-cf11109fd06a None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:23.139479 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388f9a8-5a10-4181-aa87-08847191cd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:23.140142 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388f9a8-5a10-4181-aa87-08847191cd08 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:23.140427 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388f9a8-5a10-4181-aa87-08847191cd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:23.140707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388f9a8-5a10-4181-aa87-08847191cd08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:23.141410 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388f9a8-5a10-4181-aa87-08847191cd08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:23.143340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388f9a8-5a10-4181-aa87-08847191cd08 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:23.229924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1360/2723] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:49:23 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:49:23.234137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8071eb4-0b72-4493-bb66-dd3bb97d8f59 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:23.246077 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8071eb4-0b72-4493-bb66-dd3bb97d8f59 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:23.305791 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4eff69-1e5b-429f-b0d8-fb8ca23b6c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:23.306696 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4eff69-1e5b-429f-b0d8-fb8ca23b6c6d admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:23.307030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4eff69-1e5b-429f-b0d8-fb8ca23b6c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:23.307314 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4eff69-1e5b-429f-b0d8-fb8ca23b6c6d admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:23.308026 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4eff69-1e5b-429f-b0d8-fb8ca23b6c6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:23.309958 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4eff69-1e5b-429f-b0d8-fb8ca23b6c6d admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:23.379563 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1364/2724] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:49:23 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:49:23.385331 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388f9a8-5a10-4181-aa87-08847191cd08 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:23.396155 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388f9a8-5a10-4181-aa87-08847191cd08 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:23.402093 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881693f-79e9-43c9-b0a8-f528c72e65af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:23.402780 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881693f-79e9-43c9-b0a8-f528c72e65af admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:23.403194 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881693f-79e9-43c9-b0a8-f528c72e65af admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:23.403469 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881693f-79e9-43c9-b0a8-f528c72e65af admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:23.404444 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7881693f-79e9-43c9-b0a8-f528c72e65af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:23.406024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7881693f-79e9-43c9-b0a8-f528c72e65af admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:23.419425 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1361/2725] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:49:23.423504 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4eff69-1e5b-429f-b0d8-fb8ca23b6c6d admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:23.434590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4eff69-1e5b-429f-b0d8-fb8ca23b6c6d admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:23.439118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d78ea3-9c6e-4f5a-a82a-26def4c9961a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:23.439918 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d78ea3-9c6e-4f5a-a82a-26def4c9961a admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:23.440198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d78ea3-9c6e-4f5a-a82a-26def4c9961a admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:23.440473 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d78ea3-9c6e-4f5a-a82a-26def4c9961a admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:23.441192 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d78ea3-9c6e-4f5a-a82a-26def4c9961a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:23.443071 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d78ea3-9c6e-4f5a-a82a-26def4c9961a admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:23.454505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1365/2726] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:49:23.458690 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881693f-79e9-43c9-b0a8-f528c72e65af admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:23.469585 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881693f-79e9-43c9-b0a8-f528c72e65af admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:23.474280 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30425eb4-8f25-4a89-8cbc-6f8f6f6f3ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:23.475239 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30425eb4-8f25-4a89-8cbc-6f8f6f6f3ed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:23.475518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30425eb4-8f25-4a89-8cbc-6f8f6f6f3ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:23.475792 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30425eb4-8f25-4a89-8cbc-6f8f6f6f3ed2 admin admin] PATH_INFO: `/v3/projects/5a32f25e40a04223983d7caea1234e91/users/02b2f2a3397d4175bb99bf2b79ebbd9b/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:23.615060 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30425eb4-8f25-4a89-8cbc-6f8f6f6f3ed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a32f25e40a04223983d7caea1234e91, user_id=02b2f2a3397d4175bb99bf2b79ebbd9b, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:23.618497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30425eb4-8f25-4a89-8cbc-6f8f6f6f3ed2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:23.676021 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1362/2727] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:49:23 2022] PUT /identity/v3/projects/5a32f25e40a04223983d7caea1234e91/users/02b2f2a3397d4175bb99bf2b79ebbd9b/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:49:23.688097 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1780e-a739-4ae6-aca8-4fd3c8b6f5fb None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:23.688386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1780e-a739-4ae6-aca8-4fd3c8b6f5fb None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:23.688668 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1780e-a739-4ae6-aca8-4fd3c8b6f5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:23.715322 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-faf1780e-a739-4ae6-aca8-4fd3c8b6f5fb None None] MFA Rules not processed for user `02b2f2a3397d4175bb99bf2b79ebbd9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:23.716527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-faf1780e-a739-4ae6-aca8-4fd3c8b6f5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:23.720388 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:23.782316 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1366/2728] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:49:23.786830 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30425eb4-8f25-4a89-8cbc-6f8f6f6f3ed2 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:23.840064 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:23.850489 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30425eb4-8f25-4a89-8cbc-6f8f6f6f3ed2 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:23.855390 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e97aaa-abfa-42d4-85e8-de28fa1fa617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:23.856064 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e97aaa-abfa-42d4-85e8-de28fa1fa617 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:23.856346 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e97aaa-abfa-42d4-85e8-de28fa1fa617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:23.856623 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e97aaa-abfa-42d4-85e8-de28fa1fa617 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:23.857337 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e97aaa-abfa-42d4-85e8-de28fa1fa617 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:23.859283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e97aaa-abfa-42d4-85e8-de28fa1fa617 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:23.957797 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1363/2729] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:49:23 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:49:23.962826 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1780e-a739-4ae6-aca8-4fd3c8b6f5fb None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:23.973381 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1780e-a739-4ae6-aca8-4fd3c8b6f5fb None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:24.029604 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd11b1-de9a-43b9-8ce3-a88e746a9297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:24.030268 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd11b1-de9a-43b9-8ce3-a88e746a9297 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:24.030551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd11b1-de9a-43b9-8ce3-a88e746a9297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:24.030827 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd11b1-de9a-43b9-8ce3-a88e746a9297 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:24.031539 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd11b1-de9a-43b9-8ce3-a88e746a9297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:24.033531 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd11b1-de9a-43b9-8ce3-a88e746a9297 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:24.119863 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1367/2730] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:49:23 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:49:24.123986 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e97aaa-abfa-42d4-85e8-de28fa1fa617 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:24.135306 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e97aaa-abfa-42d4-85e8-de28fa1fa617 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:24.140208 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44051fb7-33bc-4911-9278-d5a6082fd89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:24.140945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44051fb7-33bc-4911-9278-d5a6082fd89a admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:24.141230 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44051fb7-33bc-4911-9278-d5a6082fd89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:24.141503 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44051fb7-33bc-4911-9278-d5a6082fd89a admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:24.142220 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44051fb7-33bc-4911-9278-d5a6082fd89a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:24.143875 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44051fb7-33bc-4911-9278-d5a6082fd89a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:24.155115 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1364/2731] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:49:24.159054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd11b1-de9a-43b9-8ce3-a88e746a9297 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:24.168984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd11b1-de9a-43b9-8ce3-a88e746a9297 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:24.173523 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319bab4-643f-46a2-b1f9-82ebd8ab6f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:24.174169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319bab4-643f-46a2-b1f9-82ebd8ab6f95 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:24.174451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319bab4-643f-46a2-b1f9-82ebd8ab6f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:24.174724 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319bab4-643f-46a2-b1f9-82ebd8ab6f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:24.175475 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319bab4-643f-46a2-b1f9-82ebd8ab6f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:24.177052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319bab4-643f-46a2-b1f9-82ebd8ab6f95 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:24.187503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1368/2732] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:49:24.191407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44051fb7-33bc-4911-9278-d5a6082fd89a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:24.202073 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44051fb7-33bc-4911-9278-d5a6082fd89a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:24.206558 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ec045-129f-4215-93e2-2afe7210ba7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:24.207459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ec045-129f-4215-93e2-2afe7210ba7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:24.207742 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ec045-129f-4215-93e2-2afe7210ba7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:24.208018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ec045-129f-4215-93e2-2afe7210ba7a admin admin] PATH_INFO: `/v3/projects/2fac7876a6df49d084e8dd5d0f882447/users/8e71ad9bc54a43c69b5488d5ebbb6ab1/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:24.222262 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ec045-129f-4215-93e2-2afe7210ba7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fac7876a6df49d084e8dd5d0f882447, user_id=8e71ad9bc54a43c69b5488d5ebbb6ab1, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:24.225031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ec045-129f-4215-93e2-2afe7210ba7a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:24.342222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1365/2733] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:49:24 2022] PUT /identity/v3/projects/2fac7876a6df49d084e8dd5d0f882447/users/8e71ad9bc54a43c69b5488d5ebbb6ab1/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:49:24.353143 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e6e5f-3ed6-43fc-8ce8-82446d621f8c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:24.353431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e6e5f-3ed6-43fc-8ce8-82446d621f8c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:24.353707 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e6e5f-3ed6-43fc-8ce8-82446d621f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:24.379846 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-536e6e5f-3ed6-43fc-8ce8-82446d621f8c None None] MFA Rules not processed for user `8e71ad9bc54a43c69b5488d5ebbb6ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:24.381604 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-536e6e5f-3ed6-43fc-8ce8-82446d621f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:24.445300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1369/2734] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:49:24.449559 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ec045-129f-4215-93e2-2afe7210ba7a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:24.512848 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ec045-129f-4215-93e2-2afe7210ba7a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:24.517543 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b29958-9851-4331-b083-98d5179a8fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:24.518198 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b29958-9851-4331-b083-98d5179a8fe0 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:24.518482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b29958-9851-4331-b083-98d5179a8fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:24.518757 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b29958-9851-4331-b083-98d5179a8fe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:24.519515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b29958-9851-4331-b083-98d5179a8fe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:24.521417 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b29958-9851-4331-b083-98d5179a8fe0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:24.613399 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1366/2735] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:49:24 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:49:24.619799 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e6e5f-3ed6-43fc-8ce8-82446d621f8c None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:24.630917 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e6e5f-3ed6-43fc-8ce8-82446d621f8c None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:24.640961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:24.692388 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eda2e9-523a-4898-a831-ef1068c6b9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:24.693077 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eda2e9-523a-4898-a831-ef1068c6b9d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:24.693861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eda2e9-523a-4898-a831-ef1068c6b9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:24.694152 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eda2e9-523a-4898-a831-ef1068c6b9d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:24.696386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eda2e9-523a-4898-a831-ef1068c6b9d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:24.696906 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eda2e9-523a-4898-a831-ef1068c6b9d0 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:24.780452 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1370/2736] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:49:24 2022] POST /identity/v3/users => generated 299 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:49:24.784598 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b29958-9851-4331-b083-98d5179a8fe0 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:24.795725 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b29958-9851-4331-b083-98d5179a8fe0 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:24.800452 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17413c0a-bea9-4b2c-b201-277089750c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:24.801110 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17413c0a-bea9-4b2c-b201-277089750c98 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:24.801394 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17413c0a-bea9-4b2c-b201-277089750c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:24.801670 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17413c0a-bea9-4b2c-b201-277089750c98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:24.802391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17413c0a-bea9-4b2c-b201-277089750c98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:24.804003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17413c0a-bea9-4b2c-b201-277089750c98 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:24.814721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1367/2737] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:49:24.823062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eda2e9-523a-4898-a831-ef1068c6b9d0 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:24.833041 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eda2e9-523a-4898-a831-ef1068c6b9d0 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:24.838264 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c911db5-d37f-4c08-904c-bd9b6cdb9c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:24.839337 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c911db5-d37f-4c08-904c-bd9b6cdb9c7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:24.840046 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c911db5-d37f-4c08-904c-bd9b6cdb9c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:24.840322 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c911db5-d37f-4c08-904c-bd9b6cdb9c7d admin admin] PATH_INFO: `/v3/projects/abb031b99bfd41389f666bd195568d4c/users/aed306ad9f594dfaaea66ae73ef9c670/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:24.855279 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c911db5-d37f-4c08-904c-bd9b6cdb9c7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb031b99bfd41389f666bd195568d4c, user_id=aed306ad9f594dfaaea66ae73ef9c670, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:24.857938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c911db5-d37f-4c08-904c-bd9b6cdb9c7d admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:24.915902 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1371/2738] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:49:24 2022] PUT /identity/v3/projects/abb031b99bfd41389f666bd195568d4c/users/aed306ad9f594dfaaea66ae73ef9c670/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:49:24.920018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17413c0a-bea9-4b2c-b201-277089750c98 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:24.930325 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17413c0a-bea9-4b2c-b201-277089750c98 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:24.985700 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f03ba7-9608-4b71-83dc-9e7baff9cd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:24.986364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f03ba7-9608-4b71-83dc-9e7baff9cd7b admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:24.986651 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f03ba7-9608-4b71-83dc-9e7baff9cd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:24.986958 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f03ba7-9608-4b71-83dc-9e7baff9cd7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:24.987679 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f03ba7-9608-4b71-83dc-9e7baff9cd7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:24.989287 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f03ba7-9608-4b71-83dc-9e7baff9cd7b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:24.999289 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1368/2739] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:49:25.003405 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c911db5-d37f-4c08-904c-bd9b6cdb9c7d admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:25.014101 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c911db5-d37f-4c08-904c-bd9b6cdb9c7d admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:25.018757 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbe9bb-b000-4a35-977c-8395ca6733ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:25.019658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbe9bb-b000-4a35-977c-8395ca6733ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:25.019935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbe9bb-b000-4a35-977c-8395ca6733ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:25.020213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbe9bb-b000-4a35-977c-8395ca6733ec admin admin] PATH_INFO: `/v3/projects/abb031b99bfd41389f666bd195568d4c/users/aed306ad9f594dfaaea66ae73ef9c670/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:25.023018 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbe9bb-b000-4a35-977c-8395ca6733ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb031b99bfd41389f666bd195568d4c, user_id=aed306ad9f594dfaaea66ae73ef9c670, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:25.025708 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbe9bb-b000-4a35-977c-8395ca6733ec admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:25.036518 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1372/2740] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:49:25 2022] PUT /identity/v3/projects/abb031b99bfd41389f666bd195568d4c/users/aed306ad9f594dfaaea66ae73ef9c670/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:49:25.047345 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77369bc-21c8-44e7-b39c-565fedf86acc None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:25.047640 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77369bc-21c8-44e7-b39c-565fedf86acc None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:25.047922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77369bc-21c8-44e7-b39c-565fedf86acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:25.071036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:25.074089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a77369bc-21c8-44e7-b39c-565fedf86acc None None] MFA Rules not processed for user `aed306ad9f594dfaaea66ae73ef9c670`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:25.075389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a77369bc-21c8-44e7-b39c-565fedf86acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:25.141037 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1369/2741] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:49:25.153249 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbe9bb-b000-4a35-977c-8395ca6733ec admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:25.155210 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-04dbe9bb-b000-4a35-977c-8395ca6733ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:25.213181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbe9bb-b000-4a35-977c-8395ca6733ec admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:25.216820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f842d-bf65-4609-8404-49a6232792f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:25.217455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f842d-bf65-4609-8404-49a6232792f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:25.217731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f842d-bf65-4609-8404-49a6232792f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:25.218003 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f842d-bf65-4609-8404-49a6232792f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:25.275430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f842d-bf65-4609-8404-49a6232792f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:25.276771 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f842d-bf65-4609-8404-49a6232792f0 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:25.285721 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1373/2742] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:49:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:49:27.207144 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77369bc-21c8-44e7-b39c-565fedf86acc None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:27.208539 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a77369bc-21c8-44e7-b39c-565fedf86acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:27.220268 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77369bc-21c8-44e7-b39c-565fedf86acc None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:27.223788 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2467d2-a86c-495d-87d8-bc36d14323e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:27.224420 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2467d2-a86c-495d-87d8-bc36d14323e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:27.224702 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2467d2-a86c-495d-87d8-bc36d14323e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:27.224976 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2467d2-a86c-495d-87d8-bc36d14323e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:27.281982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2467d2-a86c-495d-87d8-bc36d14323e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:27.283485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2467d2-a86c-495d-87d8-bc36d14323e5 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:27.293632 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1370/2743] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:49:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:49:28.558637 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f842d-bf65-4609-8404-49a6232792f0 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:28.567298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f842d-bf65-4609-8404-49a6232792f0 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:28.572015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0e81d-56c8-4fa4-bfb8-45428b689faf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:28.572661 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0e81d-56c8-4fa4-bfb8-45428b689faf service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:28.572946 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0e81d-56c8-4fa4-bfb8-45428b689faf service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:28.573223 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0e81d-56c8-4fa4-bfb8-45428b689faf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:28.582376 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0e81d-56c8-4fa4-bfb8-45428b689faf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:28.583825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0e81d-56c8-4fa4-bfb8-45428b689faf service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:28.593439 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1374/2744] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:49:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:49:33.122928 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2467d2-a86c-495d-87d8-bc36d14323e5 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:33.202463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2467d2-a86c-495d-87d8-bc36d14323e5 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:33.214775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae24379-cefd-4ff4-879a-179e97426848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:33.214775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae24379-cefd-4ff4-879a-179e97426848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:33.214775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae24379-cefd-4ff4-879a-179e97426848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:33.214775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae24379-cefd-4ff4-879a-179e97426848 admin admin] PATH_INFO: `/v3/users/02b2f2a3397d4175bb99bf2b79ebbd9b` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:33.215312 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae24379-cefd-4ff4-879a-179e97426848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02b2f2a3397d4175bb99bf2b79ebbd9b)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:33.215312 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae24379-cefd-4ff4-879a-179e97426848 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:33.234838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:33.440717 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5ae24379-cefd-4ff4-879a-179e97426848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02b2f2a3397d4175bb99bf2b79ebbd9b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:33.544515 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5ae24379-cefd-4ff4-879a-179e97426848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02b2f2a3397d4175bb99bf2b79ebbd9b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:33.545143 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5ae24379-cefd-4ff4-879a-179e97426848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02b2f2a3397d4175bb99bf2b79ebbd9b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:33.545781 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5ae24379-cefd-4ff4-879a-179e97426848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02b2f2a3397d4175bb99bf2b79ebbd9b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:33.553541 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5ae24379-cefd-4ff4-879a-179e97426848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02b2f2a3397d4175bb99bf2b79ebbd9b'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:33.572428 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1371/2745] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:49:33 2022] DELETE /identity/v3/users/02b2f2a3397d4175bb99bf2b79ebbd9b => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:49:33.577053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfb3c5-3274-4f64-988f-cd5ea39c4cc4 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:33.577568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfb3c5-3274-4f64-988f-cd5ea39c4cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:33.577650 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfb3c5-3274-4f64-988f-cd5ea39c4cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:33.589509 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:33.596157 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-c1cfb3c5-3274-4f64-988f-cd5ea39c4cc4 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:33.597355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c1cfb3c5-3274-4f64-988f-cd5ea39c4cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:33.655004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1375/2746] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:49:33.665025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae24379-cefd-4ff4-879a-179e97426848 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:33.666407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5ae24379-cefd-4ff4-879a-179e97426848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:33.728580 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae24379-cefd-4ff4-879a-179e97426848 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:33.732700 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152c8fc-e5d2-4910-9128-2d707617ec8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:33.733336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152c8fc-e5d2-4910-9128-2d707617ec8f service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:33.733616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152c8fc-e5d2-4910-9128-2d707617ec8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:33.733888 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152c8fc-e5d2-4910-9128-2d707617ec8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:33.743193 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152c8fc-e5d2-4910-9128-2d707617ec8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:33.744611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152c8fc-e5d2-4910-9128-2d707617ec8f service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:33.753961 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1372/2747] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:49:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 16 15:49:34.140568 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cfb3c5-3274-4f64-988f-cd5ea39c4cc4 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:34.142006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c1cfb3c5-3274-4f64-988f-cd5ea39c4cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:34.156111 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cfb3c5-3274-4f64-988f-cd5ea39c4cc4 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:34.161225 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebd22b-5f7a-4e24-966e-d2b77f37efa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:34.162254 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebd22b-5f7a-4e24-966e-d2b77f37efa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:34.162599 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebd22b-5f7a-4e24-966e-d2b77f37efa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:34.162794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebd22b-5f7a-4e24-966e-d2b77f37efa5 admin admin] PATH_INFO: `/v3/projects/5a32f25e40a04223983d7caea1234e91` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:34.164006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebd22b-5f7a-4e24-966e-d2b77f37efa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a32f25e40a04223983d7caea1234e91)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:34.165936 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebd22b-5f7a-4e24-966e-d2b77f37efa5 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:34.269705 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-acebd22b-5f7a-4e24-966e-d2b77f37efa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a32f25e40a04223983d7caea1234e91', 'request_id': 'req-acebd22b-5f7a-4e24-966e-d2b77f37efa5'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:34.275363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-acebd22b-5f7a-4e24-966e-d2b77f37efa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a32f25e40a04223983d7caea1234e91', 'request_id': 'req-acebd22b-5f7a-4e24-966e-d2b77f37efa5'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:34.282655 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-acebd22b-5f7a-4e24-966e-d2b77f37efa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a32f25e40a04223983d7caea1234e91', 'request_id': 'req-acebd22b-5f7a-4e24-966e-d2b77f37efa5'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:34.343503 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-acebd22b-5f7a-4e24-966e-d2b77f37efa5 admin admin] The token cache is being invalidate because project 5a32f25e40a04223983d7caea1234e91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:49:34.343826 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-acebd22b-5f7a-4e24-966e-d2b77f37efa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:34.344456 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-acebd22b-5f7a-4e24-966e-d2b77f37efa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:34.346353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1376/2748] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:49:34 2022] DELETE /identity/v3/projects/5a32f25e40a04223983d7caea1234e91 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:49:34.350155 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152c8fc-e5d2-4910-9128-2d707617ec8f service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:34.360686 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152c8fc-e5d2-4910-9128-2d707617ec8f service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:34.416826 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebe715-cb32-449e-aeed-a518c5c34ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:34.417698 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebe715-cb32-449e-aeed-a518c5c34ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:34.417979 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebe715-cb32-449e-aeed-a518c5c34ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:34.418258 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebe715-cb32-449e-aeed-a518c5c34ec1 admin admin] PATH_INFO: `/v3/users/8e71ad9bc54a43c69b5488d5ebbb6ab1` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:34.419478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebe715-cb32-449e-aeed-a518c5c34ec1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e71ad9bc54a43c69b5488d5ebbb6ab1)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:34.421386 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebe715-cb32-449e-aeed-a518c5c34ec1 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:34.598630 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:34.606615 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-81ebe715-cb32-449e-aeed-a518c5c34ec1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e71ad9bc54a43c69b5488d5ebbb6ab1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:34.690248 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-81ebe715-cb32-449e-aeed-a518c5c34ec1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e71ad9bc54a43c69b5488d5ebbb6ab1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:34.690860 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-81ebe715-cb32-449e-aeed-a518c5c34ec1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e71ad9bc54a43c69b5488d5ebbb6ab1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:34.691487 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-81ebe715-cb32-449e-aeed-a518c5c34ec1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e71ad9bc54a43c69b5488d5ebbb6ab1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:34.698812 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-81ebe715-cb32-449e-aeed-a518c5c34ec1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e71ad9bc54a43c69b5488d5ebbb6ab1'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:34.717178 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1373/2749] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:49:34 2022] DELETE /identity/v3/users/8e71ad9bc54a43c69b5488d5ebbb6ab1 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:49:35.133773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebd22b-5f7a-4e24-966e-d2b77f37efa5 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:35.135185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-acebd22b-5f7a-4e24-966e-d2b77f37efa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:35.155160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebd22b-5f7a-4e24-966e-d2b77f37efa5 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:35.210927 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8b049-7ad3-4339-a557-cf90d2924454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:35.211780 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8b049-7ad3-4339-a557-cf90d2924454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:35.212058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8b049-7ad3-4339-a557-cf90d2924454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:35.212328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8b049-7ad3-4339-a557-cf90d2924454 admin admin] PATH_INFO: `/v3/projects/2fac7876a6df49d084e8dd5d0f882447` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:35.213486 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8b049-7ad3-4339-a557-cf90d2924454 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fac7876a6df49d084e8dd5d0f882447)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:35.216345 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8b049-7ad3-4339-a557-cf90d2924454 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:35.337990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:35.350383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-abe8b049-7ad3-4339-a557-cf90d2924454 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fac7876a6df49d084e8dd5d0f882447', 'request_id': 'req-abe8b049-7ad3-4339-a557-cf90d2924454'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:35.356319 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-abe8b049-7ad3-4339-a557-cf90d2924454 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fac7876a6df49d084e8dd5d0f882447', 'request_id': 'req-abe8b049-7ad3-4339-a557-cf90d2924454'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:35.362421 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-abe8b049-7ad3-4339-a557-cf90d2924454 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fac7876a6df49d084e8dd5d0f882447', 'request_id': 'req-abe8b049-7ad3-4339-a557-cf90d2924454'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:35.455710 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-abe8b049-7ad3-4339-a557-cf90d2924454 admin admin] The token cache is being invalidate because project 2fac7876a6df49d084e8dd5d0f882447 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:49:35.455956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-abe8b049-7ad3-4339-a557-cf90d2924454 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:35.456574 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-abe8b049-7ad3-4339-a557-cf90d2924454 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:35.458180 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1377/2750] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:49:35 2022] DELETE /identity/v3/projects/2fac7876a6df49d084e8dd5d0f882447 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:49:35.462238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebe715-cb32-449e-aeed-a518c5c34ec1 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:35.463662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-81ebe715-cb32-449e-aeed-a518c5c34ec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:35.476656 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebe715-cb32-449e-aeed-a518c5c34ec1 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:35.533058 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002c011-bbbc-4bb3-85ec-93d4913fafdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:35.533920 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002c011-bbbc-4bb3-85ec-93d4913fafdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:35.534198 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002c011-bbbc-4bb3-85ec-93d4913fafdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:35.534471 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002c011-bbbc-4bb3-85ec-93d4913fafdd admin admin] PATH_INFO: `/v3/users/aed306ad9f594dfaaea66ae73ef9c670` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:35.535677 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002c011-bbbc-4bb3-85ec-93d4913fafdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=aed306ad9f594dfaaea66ae73ef9c670)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:35.537581 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002c011-bbbc-4bb3-85ec-93d4913fafdd admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:35.685696 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1002c011-bbbc-4bb3-85ec-93d4913fafdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aed306ad9f594dfaaea66ae73ef9c670'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:35.760211 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1002c011-bbbc-4bb3-85ec-93d4913fafdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aed306ad9f594dfaaea66ae73ef9c670'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:35.760806 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1002c011-bbbc-4bb3-85ec-93d4913fafdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aed306ad9f594dfaaea66ae73ef9c670'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:35.761903 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1002c011-bbbc-4bb3-85ec-93d4913fafdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aed306ad9f594dfaaea66ae73ef9c670'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:35.769073 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-1002c011-bbbc-4bb3-85ec-93d4913fafdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aed306ad9f594dfaaea66ae73ef9c670'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:35.787048 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1374/2751] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:49:35 2022] DELETE /identity/v3/users/aed306ad9f594dfaaea66ae73ef9c670 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:49:36.171528 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8b049-7ad3-4339-a557-cf90d2924454 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:36.172962 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-abe8b049-7ad3-4339-a557-cf90d2924454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:36.186587 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8b049-7ad3-4339-a557-cf90d2924454 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:36.245185 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22580a-acea-4ae8-a7b4-68f3dea75cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:36.246767 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22580a-acea-4ae8-a7b4-68f3dea75cd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:36.247089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22580a-acea-4ae8-a7b4-68f3dea75cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:36.250363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22580a-acea-4ae8-a7b4-68f3dea75cd8 admin admin] PATH_INFO: `/v3/projects/abb031b99bfd41389f666bd195568d4c` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:36.250363 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22580a-acea-4ae8-a7b4-68f3dea75cd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abb031b99bfd41389f666bd195568d4c)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:36.250604 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22580a-acea-4ae8-a7b4-68f3dea75cd8 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:36.382008 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7f22580a-acea-4ae8-a7b4-68f3dea75cd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abb031b99bfd41389f666bd195568d4c', 'request_id': 'req-7f22580a-acea-4ae8-a7b4-68f3dea75cd8'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:36.387599 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7f22580a-acea-4ae8-a7b4-68f3dea75cd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abb031b99bfd41389f666bd195568d4c', 'request_id': 'req-7f22580a-acea-4ae8-a7b4-68f3dea75cd8'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:36.393666 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7f22580a-acea-4ae8-a7b4-68f3dea75cd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abb031b99bfd41389f666bd195568d4c', 'request_id': 'req-7f22580a-acea-4ae8-a7b4-68f3dea75cd8'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:36.462796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7f22580a-acea-4ae8-a7b4-68f3dea75cd8 admin admin] The token cache is being invalidate because project abb031b99bfd41389f666bd195568d4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:49:36.463146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7f22580a-acea-4ae8-a7b4-68f3dea75cd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:36.463764 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7f22580a-acea-4ae8-a7b4-68f3dea75cd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:36.465283 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1378/2752] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:49:36 2022] DELETE /identity/v3/projects/abb031b99bfd41389f666bd195568d4c => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:49:36.470470 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f2f03-2b07-4beb-a7ed-b903f9a1ca30 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:36.470963 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f2f03-2b07-4beb-a7ed-b903f9a1ca30 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:36.471053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f2f03-2b07-4beb-a7ed-b903f9a1ca30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:36.489544 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-240f2f03-2b07-4beb-a7ed-b903f9a1ca30 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:36.490732 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-240f2f03-2b07-4beb-a7ed-b903f9a1ca30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:36.519973 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:36.552383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1375/2753] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:49:36.557753 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61cc902-6bd7-4626-a471-bb52e59ff4e7 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:36.558049 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61cc902-6bd7-4626-a471-bb52e59ff4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:36.559065 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61cc902-6bd7-4626-a471-bb52e59ff4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:36.577300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-b61cc902-6bd7-4626-a471-bb52e59ff4e7 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:36.578558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b61cc902-6bd7-4626-a471-bb52e59ff4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:36.586329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1379/2754] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:49:36.591404 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f2f03-2b07-4beb-a7ed-b903f9a1ca30 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:36.600770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f2f03-2b07-4beb-a7ed-b903f9a1ca30 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:36.606061 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a86f8-130e-4ece-a4e9-7dcffff80e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:36.606713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a86f8-130e-4ece-a4e9-7dcffff80e4d admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:36.607020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a86f8-130e-4ece-a4e9-7dcffff80e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:36.607293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a86f8-130e-4ece-a4e9-7dcffff80e4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:36.608007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a86f8-130e-4ece-a4e9-7dcffff80e4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:49:36.608307 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a86f8-130e-4ece-a4e9-7dcffff80e4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:36.609888 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614a86f8-130e-4ece-a4e9-7dcffff80e4d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:36.621735 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1376/2755] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:49:36.625872 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61cc902-6bd7-4626-a471-bb52e59ff4e7 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:36.634950 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61cc902-6bd7-4626-a471-bb52e59ff4e7 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:36.639588 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3994a3-eead-40c8-98d8-76cdfaffdfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:36.640216 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3994a3-eead-40c8-98d8-76cdfaffdfcd admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:36.640491 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3994a3-eead-40c8-98d8-76cdfaffdfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:36.640757 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3994a3-eead-40c8-98d8-76cdfaffdfcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:36.641441 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3994a3-eead-40c8-98d8-76cdfaffdfcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:36.643346 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3994a3-eead-40c8-98d8-76cdfaffdfcd admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:36.719272 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1380/2756] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:49:36 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:49:36.723501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a86f8-130e-4ece-a4e9-7dcffff80e4d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:36.734242 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a86f8-130e-4ece-a4e9-7dcffff80e4d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:36.790656 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ff801-71d5-451a-a79f-c686c59179f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:36.791348 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ff801-71d5-451a-a79f-c686c59179f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:36.791630 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ff801-71d5-451a-a79f-c686c59179f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:36.791908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ff801-71d5-451a-a79f-c686c59179f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:36.792588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ff801-71d5-451a-a79f-c686c59179f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:36.795212 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ff801-71d5-451a-a79f-c686c59179f4 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:36.893051 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1377/2757] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:49:36 2022] POST /identity/v3/users => generated 292 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:49:36.897152 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3994a3-eead-40c8-98d8-76cdfaffdfcd admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:36.908938 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3994a3-eead-40c8-98d8-76cdfaffdfcd admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:36.914093 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ce378-83e5-483a-86e9-c6167c54e7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:36.915018 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ce378-83e5-483a-86e9-c6167c54e7fc admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:36.915309 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ce378-83e5-483a-86e9-c6167c54e7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:36.915584 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ce378-83e5-483a-86e9-c6167c54e7fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:36.916309 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ce378-83e5-483a-86e9-c6167c54e7fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:36.918588 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ce378-83e5-483a-86e9-c6167c54e7fc admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:36.931683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1381/2758] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:49:36.935789 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ff801-71d5-451a-a79f-c686c59179f4 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:36.946241 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ff801-71d5-451a-a79f-c686c59179f4 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:36.951297 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af5166-2090-4eaf-ba91-856d57ad925e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:36.955068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af5166-2090-4eaf-ba91-856d57ad925e admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:36.955068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af5166-2090-4eaf-ba91-856d57ad925e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:36.955068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af5166-2090-4eaf-ba91-856d57ad925e admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:36.955068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af5166-2090-4eaf-ba91-856d57ad925e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:36.955068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af5166-2090-4eaf-ba91-856d57ad925e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:36.966001 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1378/2759] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:49:36.970111 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ce378-83e5-483a-86e9-c6167c54e7fc admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:36.981678 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ce378-83e5-483a-86e9-c6167c54e7fc admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:36.986551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40025f15-42bf-4a53-851b-eb1edc248d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:36.987460 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40025f15-42bf-4a53-851b-eb1edc248d2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:36.987992 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40025f15-42bf-4a53-851b-eb1edc248d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:36.988197 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40025f15-42bf-4a53-851b-eb1edc248d2c admin admin] PATH_INFO: `/v3/projects/d5509d64a1aa472095ad35da222f9963/users/745d930ebbab403c9e974f47d3719035/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:36.997176 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:37.004132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40025f15-42bf-4a53-851b-eb1edc248d2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5509d64a1aa472095ad35da222f9963, user_id=745d930ebbab403c9e974f47d3719035, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:37.007056 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40025f15-42bf-4a53-851b-eb1edc248d2c admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:37.069776 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1382/2760] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:49:36 2022] PUT /identity/v3/projects/d5509d64a1aa472095ad35da222f9963/users/745d930ebbab403c9e974f47d3719035/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:49:37.075835 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76741bdc-db1b-4128-b9e4-f376a209c862 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:37.076124 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76741bdc-db1b-4128-b9e4-f376a209c862 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:37.076542 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76741bdc-db1b-4128-b9e4-f376a209c862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:37.095181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-76741bdc-db1b-4128-b9e4-f376a209c862 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:37.096501 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-76741bdc-db1b-4128-b9e4-f376a209c862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:37.157705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1379/2761] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:49:37.167262 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40025f15-42bf-4a53-851b-eb1edc248d2c admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:37.169502 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-40025f15-42bf-4a53-851b-eb1edc248d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:37.234205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40025f15-42bf-4a53-851b-eb1edc248d2c admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:37.237919 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c065b-90ee-4cfc-ab41-d7444708376e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:37.238565 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c065b-90ee-4cfc-ab41-d7444708376e service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:37.238842 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c065b-90ee-4cfc-ab41-d7444708376e service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:37.239143 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c065b-90ee-4cfc-ab41-d7444708376e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:37.248169 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c065b-90ee-4cfc-ab41-d7444708376e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:37.249577 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c065b-90ee-4cfc-ab41-d7444708376e service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:37.258825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1383/2762] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:49:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:49:42.778094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61897d3c-d4ed-4f9d-a762-4a28189e55c7 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:42.778870 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61897d3c-d4ed-4f9d-a762-4a28189e55c7 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:42.779067 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61897d3c-d4ed-4f9d-a762-4a28189e55c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:42.808493 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-61897d3c-d4ed-4f9d-a762-4a28189e55c7 None None] MFA Rules not processed for user `745d930ebbab403c9e974f47d3719035`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:42.811429 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-61897d3c-d4ed-4f9d-a762-4a28189e55c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:42.833287 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c065b-90ee-4cfc-ab41-d7444708376e service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:42.835351 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-1b7c065b-90ee-4cfc-ab41-d7444708376e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:42.860327 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:42.863178 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c065b-90ee-4cfc-ab41-d7444708376e service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:42.877335 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90af2a4-4b8f-4838-b3ed-055f125a8bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:42.878341 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90af2a4-4b8f-4838-b3ed-055f125a8bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:42.878625 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90af2a4-4b8f-4838-b3ed-055f125a8bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:42.879284 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90af2a4-4b8f-4838-b3ed-055f125a8bbe admin admin] PATH_INFO: `/v3/users/2139919b73cf497f855338b8f90fcfca` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:42.880925 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90af2a4-4b8f-4838-b3ed-055f125a8bbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=2139919b73cf497f855338b8f90fcfca)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:42.884364 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90af2a4-4b8f-4838-b3ed-055f125a8bbe admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:42.925543 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1380/2763] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:49:42.929862 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61897d3c-d4ed-4f9d-a762-4a28189e55c7 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:42.943051 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61897d3c-d4ed-4f9d-a762-4a28189e55c7 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:42.947828 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc36a01-1945-4058-9f74-b9753322a710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:42.948502 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc36a01-1945-4058-9f74-b9753322a710 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:42.950003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc36a01-1945-4058-9f74-b9753322a710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:42.950003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc36a01-1945-4058-9f74-b9753322a710 admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:42.950003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc36a01-1945-4058-9f74-b9753322a710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:42.952093 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc36a01-1945-4058-9f74-b9753322a710 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:43.054471 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1381/2764] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:49:42 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:49:43.060225 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc36a01-1945-4058-9f74-b9753322a710 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:43.097903 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d90af2a4-4b8f-4838-b3ed-055f125a8bbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2139919b73cf497f855338b8f90fcfca'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:43.139907 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc36a01-1945-4058-9f74-b9753322a710 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:43.145228 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b8dd8d-06fa-4836-bece-82bfdbbc6bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:43.145695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8dd8d-06fa-4836-bece-82bfdbbc6bc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:43.146020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8dd8d-06fa-4836-bece-82bfdbbc6bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:43.146248 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b8dd8d-06fa-4836-bece-82bfdbbc6bc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:43.147236 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b8dd8d-06fa-4836-bece-82bfdbbc6bc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:43.149175 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b8dd8d-06fa-4836-bece-82bfdbbc6bc7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:43.158772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d90af2a4-4b8f-4838-b3ed-055f125a8bbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2139919b73cf497f855338b8f90fcfca'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:43.167198 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d90af2a4-4b8f-4838-b3ed-055f125a8bbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2139919b73cf497f855338b8f90fcfca'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:43.186385 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d90af2a4-4b8f-4838-b3ed-055f125a8bbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2139919b73cf497f855338b8f90fcfca'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:43.187184 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-d90af2a4-4b8f-4838-b3ed-055f125a8bbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2139919b73cf497f855338b8f90fcfca'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:43.193995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1384/2765] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:49:42 2022] DELETE /identity/v3/users/2139919b73cf497f855338b8f90fcfca => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:49:43.198520 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc4742-46de-48b7-b3c6-bc1cf9ae6a27 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:43.199678 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc4742-46de-48b7-b3c6-bc1cf9ae6a27 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:43.199960 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc4742-46de-48b7-b3c6-bc1cf9ae6a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:43.220606 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-5ffc4742-46de-48b7-b3c6-bc1cf9ae6a27 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:43.221834 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5ffc4742-46de-48b7-b3c6-bc1cf9ae6a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:43.246203 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1382/2766] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:49:43 2022] POST /identity/v3/users => generated 299 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:49:43.249966 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b8dd8d-06fa-4836-bece-82bfdbbc6bc7 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:43.254340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a8b8dd8d-06fa-4836-bece-82bfdbbc6bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:43.274000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b8dd8d-06fa-4836-bece-82bfdbbc6bc7 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:43.304356 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1385/2767] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:49:43.314862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc4742-46de-48b7-b3c6-bc1cf9ae6a27 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:43.316385 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5ffc4742-46de-48b7-b3c6-bc1cf9ae6a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:43.321882 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:43.345869 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13add6a5-98dc-42d2-b870-11ce8ea9578b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:43.346645 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13add6a5-98dc-42d2-b870-11ce8ea9578b admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:43.347292 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13add6a5-98dc-42d2-b870-11ce8ea9578b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:43.347451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13add6a5-98dc-42d2-b870-11ce8ea9578b admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:43.348248 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13add6a5-98dc-42d2-b870-11ce8ea9578b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:43.350109 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13add6a5-98dc-42d2-b870-11ce8ea9578b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:43.362641 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1383/2768] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:49:43.368174 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13add6a5-98dc-42d2-b870-11ce8ea9578b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:43.384313 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13add6a5-98dc-42d2-b870-11ce8ea9578b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:43.389325 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc4742-46de-48b7-b3c6-bc1cf9ae6a27 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:43.390171 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cfecc-5a11-4824-a3e7-ae6ed5016bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:43.391099 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cfecc-5a11-4824-a3e7-ae6ed5016bdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:43.391384 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cfecc-5a11-4824-a3e7-ae6ed5016bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:43.391663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5cfecc-5a11-4824-a3e7-ae6ed5016bdf admin admin] PATH_INFO: `/v3/projects/fe4f9e430c484aa9aa4fd8e27db0918e/users/365f84b02d354f5bbfc42a46b33b4825/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:43.394414 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11de4b5-bbed-4622-96d3-80a478567cc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:43.395928 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11de4b5-bbed-4622-96d3-80a478567cc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:43.395928 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11de4b5-bbed-4622-96d3-80a478567cc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:43.395928 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11de4b5-bbed-4622-96d3-80a478567cc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:43.407229 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11de4b5-bbed-4622-96d3-80a478567cc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:43.408691 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11de4b5-bbed-4622-96d3-80a478567cc2 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:43.410387 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5cfecc-5a11-4824-a3e7-ae6ed5016bdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe4f9e430c484aa9aa4fd8e27db0918e, user_id=365f84b02d354f5bbfc42a46b33b4825, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:43.413366 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5cfecc-5a11-4824-a3e7-ae6ed5016bdf admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:43.422016 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1386/2769] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:49:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:49:43.504759 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1384/2770] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:49:43 2022] PUT /identity/v3/projects/fe4f9e430c484aa9aa4fd8e27db0918e/users/365f84b02d354f5bbfc42a46b33b4825/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:49:43.508873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11de4b5-bbed-4622-96d3-80a478567cc2 service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:43.521469 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11de4b5-bbed-4622-96d3-80a478567cc2 service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:43.605931 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caede3a-fa70-4b26-9216-bda4b403bc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:43.606709 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caede3a-fa70-4b26-9216-bda4b403bc08 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:43.607093 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caede3a-fa70-4b26-9216-bda4b403bc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:43.607438 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caede3a-fa70-4b26-9216-bda4b403bc08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:43.608233 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caede3a-fa70-4b26-9216-bda4b403bc08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:43.609947 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caede3a-fa70-4b26-9216-bda4b403bc08 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:43.624244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1387/2771] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:49:43.628072 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cfecc-5a11-4824-a3e7-ae6ed5016bdf admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:43.639735 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5cfecc-5a11-4824-a3e7-ae6ed5016bdf admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:43.644461 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334dcf40-dabe-42ae-bcd2-c71f3a5eff89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:43.645383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334dcf40-dabe-42ae-bcd2-c71f3a5eff89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:43.645663 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334dcf40-dabe-42ae-bcd2-c71f3a5eff89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:43.645942 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334dcf40-dabe-42ae-bcd2-c71f3a5eff89 admin admin] PATH_INFO: `/v3/projects/fe4f9e430c484aa9aa4fd8e27db0918e/users/365f84b02d354f5bbfc42a46b33b4825/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:43.648020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334dcf40-dabe-42ae-bcd2-c71f3a5eff89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe4f9e430c484aa9aa4fd8e27db0918e, user_id=365f84b02d354f5bbfc42a46b33b4825, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:43.650824 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334dcf40-dabe-42ae-bcd2-c71f3a5eff89 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:43.655969 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:43.671599 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1385/2772] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:49:43 2022] PUT /identity/v3/projects/fe4f9e430c484aa9aa4fd8e27db0918e/users/365f84b02d354f5bbfc42a46b33b4825/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:49:43.890132 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caede3a-fa70-4b26-9216-bda4b403bc08 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:43.893620 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3caede3a-fa70-4b26-9216-bda4b403bc08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:43.908081 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caede3a-fa70-4b26-9216-bda4b403bc08 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:43.975834 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d47ae-a459-439c-be64-a99722b53436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:43.976698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d47ae-a459-439c-be64-a99722b53436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:43.977081 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d47ae-a459-439c-be64-a99722b53436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:43.977297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d47ae-a459-439c-be64-a99722b53436 admin admin] PATH_INFO: `/v3/projects/5c0bd5c1dd8b4d88be1ded374b9bceef` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:43.978834 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d47ae-a459-439c-be64-a99722b53436 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c0bd5c1dd8b4d88be1ded374b9bceef)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:43.982070 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d47ae-a459-439c-be64-a99722b53436 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:44.092717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:44.096620 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-cd7d47ae-a459-439c-be64-a99722b53436 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c0bd5c1dd8b4d88be1ded374b9bceef', 'request_id': 'req-cd7d47ae-a459-439c-be64-a99722b53436'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:44.102639 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-cd7d47ae-a459-439c-be64-a99722b53436 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c0bd5c1dd8b4d88be1ded374b9bceef', 'request_id': 'req-cd7d47ae-a459-439c-be64-a99722b53436'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:44.110251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-cd7d47ae-a459-439c-be64-a99722b53436 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c0bd5c1dd8b4d88be1ded374b9bceef', 'request_id': 'req-cd7d47ae-a459-439c-be64-a99722b53436'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:44.183806 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-cd7d47ae-a459-439c-be64-a99722b53436 admin admin] The token cache is being invalidate because project 5c0bd5c1dd8b4d88be1ded374b9bceef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:49:44.184153 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-cd7d47ae-a459-439c-be64-a99722b53436 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:44.184809 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-cd7d47ae-a459-439c-be64-a99722b53436 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:44.186344 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1388/2773] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:49:43 2022] DELETE /identity/v3/projects/5c0bd5c1dd8b4d88be1ded374b9bceef => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:49:44.191459 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fef68-ec40-4123-a116-1eca3b9bb7a2 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:44.191902 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fef68-ec40-4123-a116-1eca3b9bb7a2 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:44.192018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fef68-ec40-4123-a116-1eca3b9bb7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:44.211674 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-260fef68-ec40-4123-a116-1eca3b9bb7a2 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:44.213192 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-260fef68-ec40-4123-a116-1eca3b9bb7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:44.279129 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1386/2774] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:49:44.287980 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04441f30-0c16-4991-baa1-b54e95c7b048 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:44.288344 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04441f30-0c16-4991-baa1-b54e95c7b048 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:44.288693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04441f30-0c16-4991-baa1-b54e95c7b048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:44.309837 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-04441f30-0c16-4991-baa1-b54e95c7b048 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:44.311423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-04441f30-0c16-4991-baa1-b54e95c7b048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:44.320360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1389/2775] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:49:44.325089 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fef68-ec40-4123-a116-1eca3b9bb7a2 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:44.335989 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fef68-ec40-4123-a116-1eca3b9bb7a2 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:44.342302 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28a88e-10c1-4a6b-96ff-fbb066979f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:44.343019 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28a88e-10c1-4a6b-96ff-fbb066979f54 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:44.343310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28a88e-10c1-4a6b-96ff-fbb066979f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:44.343588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28a88e-10c1-4a6b-96ff-fbb066979f54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:44.344340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a28a88e-10c1-4a6b-96ff-fbb066979f54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:49:44.344650 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a28a88e-10c1-4a6b-96ff-fbb066979f54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:44.346557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a28a88e-10c1-4a6b-96ff-fbb066979f54 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:44.366649 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04441f30-0c16-4991-baa1-b54e95c7b048 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:44.371586 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1387/2776] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:49:44.377014 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04441f30-0c16-4991-baa1-b54e95c7b048 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:44.383015 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256b543-c00a-4544-addb-d4063e7a6ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:44.383846 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256b543-c00a-4544-addb-d4063e7a6ce8 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:44.384137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256b543-c00a-4544-addb-d4063e7a6ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:44.384412 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256b543-c00a-4544-addb-d4063e7a6ce8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:44.385126 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256b543-c00a-4544-addb-d4063e7a6ce8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:44.387290 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256b543-c00a-4544-addb-d4063e7a6ce8 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:44.486013 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1390/2777] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:49:44 2022] POST /identity/v3/projects => generated 399 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:49:44.490078 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28a88e-10c1-4a6b-96ff-fbb066979f54 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:44.508605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28a88e-10c1-4a6b-96ff-fbb066979f54 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:44.594192 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f38f14-5ac7-414e-801c-ea45b9834d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:44.595726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f38f14-5ac7-414e-801c-ea45b9834d50 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:44.596012 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f38f14-5ac7-414e-801c-ea45b9834d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:44.596328 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f38f14-5ac7-414e-801c-ea45b9834d50 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:44.597115 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f38f14-5ac7-414e-801c-ea45b9834d50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:44.599285 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f38f14-5ac7-414e-801c-ea45b9834d50 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:44.723992 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1388/2778] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:49:44 2022] POST /identity/v3/users => generated 300 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:49:44.728060 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256b543-c00a-4544-addb-d4063e7a6ce8 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:44.738661 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256b543-c00a-4544-addb-d4063e7a6ce8 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:44.743790 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b8ca0-72a2-4434-9db3-924b3015ed5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:44.744512 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b8ca0-72a2-4434-9db3-924b3015ed5b admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:44.744792 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b8ca0-72a2-4434-9db3-924b3015ed5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:44.745069 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694b8ca0-72a2-4434-9db3-924b3015ed5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:44.745935 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b8ca0-72a2-4434-9db3-924b3015ed5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:44.747721 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694b8ca0-72a2-4434-9db3-924b3015ed5b admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:44.758527 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1391/2779] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:49:44.763027 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f38f14-5ac7-414e-801c-ea45b9834d50 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:44.773637 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f38f14-5ac7-414e-801c-ea45b9834d50 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:44.779770 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25b90d-f544-45eb-8bed-5aa1aabaf1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:44.780449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25b90d-f544-45eb-8bed-5aa1aabaf1cd admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:44.780804 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25b90d-f544-45eb-8bed-5aa1aabaf1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:44.781149 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25b90d-f544-45eb-8bed-5aa1aabaf1cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:44.781982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab25b90d-f544-45eb-8bed-5aa1aabaf1cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:44.783695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab25b90d-f544-45eb-8bed-5aa1aabaf1cd admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:44.794511 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1389/2780] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:49:44.799014 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b8ca0-72a2-4434-9db3-924b3015ed5b admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:44.811010 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694b8ca0-72a2-4434-9db3-924b3015ed5b admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:44.816237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb015de-01c1-448c-8720-1bf194e5c6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:44.816965 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb015de-01c1-448c-8720-1bf194e5c6f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:44.817315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb015de-01c1-448c-8720-1bf194e5c6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:44.817703 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb015de-01c1-448c-8720-1bf194e5c6f1 admin admin] PATH_INFO: `/v3/projects/615dd52181464f35a8e6d2247c3668f7/users/6db951fe182a4a4da846055cf36f8821/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:44.836420 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb015de-01c1-448c-8720-1bf194e5c6f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=615dd52181464f35a8e6d2247c3668f7, user_id=6db951fe182a4a4da846055cf36f8821, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:44.840080 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb015de-01c1-448c-8720-1bf194e5c6f1 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:44.915481 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1392/2781] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:49:44 2022] PUT /identity/v3/projects/615dd52181464f35a8e6d2247c3668f7/users/6db951fe182a4a4da846055cf36f8821/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:49:44.926393 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50489d95-38cc-4d46-a4a8-e91936be3cc4 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:44.926680 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50489d95-38cc-4d46-a4a8-e91936be3cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:44.927006 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50489d95-38cc-4d46-a4a8-e91936be3cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:44.957519 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-50489d95-38cc-4d46-a4a8-e91936be3cc4 None None] MFA Rules not processed for user `6db951fe182a4a4da846055cf36f8821`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:44.958749 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-50489d95-38cc-4d46-a4a8-e91936be3cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:45.017852 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:45.035628 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1390/2782] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:49:45.048052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb015de-01c1-448c-8720-1bf194e5c6f1 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:45.050106 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ffb015de-01c1-448c-8720-1bf194e5c6f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:45.125147 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb015de-01c1-448c-8720-1bf194e5c6f1 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:45.129029 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0caa2bb-b26a-4076-ac71-9c5d8816cfe4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:45.129661 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0caa2bb-b26a-4076-ac71-9c5d8816cfe4 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:45.129992 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0caa2bb-b26a-4076-ac71-9c5d8816cfe4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:45.134833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0caa2bb-b26a-4076-ac71-9c5d8816cfe4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:45.146294 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0caa2bb-b26a-4076-ac71-9c5d8816cfe4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:45.146966 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0caa2bb-b26a-4076-ac71-9c5d8816cfe4 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:45.161561 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1393/2783] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:49:45 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:49:49.247941 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf1b6b-bb57-410f-920c-69be71c140db None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:49.248468 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf1b6b-bb57-410f-920c-69be71c140db None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:49.248468 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf1b6b-bb57-410f-920c-69be71c140db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:49.277858 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-36bf1b6b-bb57-410f-920c-69be71c140db None None] MFA Rules not processed for user `365f84b02d354f5bbfc42a46b33b4825`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:49.279360 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-36bf1b6b-bb57-410f-920c-69be71c140db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:49.350830 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1391/2784] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:49:49.362840 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0caa2bb-b26a-4076-ac71-9c5d8816cfe4 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:49.364277 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-d0caa2bb-b26a-4076-ac71-9c5d8816cfe4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:49.431301 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0caa2bb-b26a-4076-ac71-9c5d8816cfe4 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:49.435629 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ccbbdf-0216-4e50-841b-a7de2b864d29 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d689162291f4f4b97e886ea20d3a8b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:49.436281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccbbdf-0216-4e50-841b-a7de2b864d29 service glance] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:49.436555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccbbdf-0216-4e50-841b-a7de2b864d29 service glance] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:49.436825 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ccbbdf-0216-4e50-841b-a7de2b864d29 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:49.438564 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a2ccbbdf-0216-4e50-841b-a7de2b864d29 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:49.465144 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:49.499400 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ccbbdf-0216-4e50-841b-a7de2b864d29 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:49.500845 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ccbbdf-0216-4e50-841b-a7de2b864d29 service glance] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:49.509881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1394/2785] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:49:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:49:49.761824 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb92a1-c306-48dc-8c5b-ddd774abfc60 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:49.762121 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb92a1-c306-48dc-8c5b-ddd774abfc60 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:49.762392 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb92a1-c306-48dc-8c5b-ddd774abfc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:49.781839 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-80bb92a1-c306-48dc-8c5b-ddd774abfc60 None None] MFA Rules not processed for user `a4f499707998403d8f833eb783599466`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:49.783352 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-80bb92a1-c306-48dc-8c5b-ddd774abfc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:49.855488 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1392/2786] 10.222.0.27 () {58 vars in 1009 bytes} [Sat Jul 16 15:49:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:49:50.012109 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ccbbdf-0216-4e50-841b-a7de2b864d29 service glance] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:50.014489 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a2ccbbdf-0216-4e50-841b-a7de2b864d29 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:50.095171 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ccbbdf-0216-4e50-841b-a7de2b864d29 service glance] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:50.100303 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b166b2-5f3d-4098-85fe-3effd0294d55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1464c83fb294a3d90e5763ef648e834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:50.100960 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b166b2-5f3d-4098-85fe-3effd0294d55 service swift] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:50.101244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b166b2-5f3d-4098-85fe-3effd0294d55 service swift] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:50.101520 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b166b2-5f3d-4098-85fe-3effd0294d55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:50.110963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b166b2-5f3d-4098-85fe-3effd0294d55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:50.112524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b166b2-5f3d-4098-85fe-3effd0294d55 service swift] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:50.122515 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1395/2787] 10.222.0.27 () {60 vars in 1384 bytes} [Sat Jul 16 15:49:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 16 15:49:54.408987 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb92a1-c306-48dc-8c5b-ddd774abfc60 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:54.551071 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb92a1-c306-48dc-8c5b-ddd774abfc60 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:54.571806 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170e9db-dc8d-4b44-81f3-3084f2343789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:54.572765 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170e9db-dc8d-4b44-81f3-3084f2343789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:54.573097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170e9db-dc8d-4b44-81f3-3084f2343789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:54.573446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170e9db-dc8d-4b44-81f3-3084f2343789 admin admin] PATH_INFO: `/v3/users/6db951fe182a4a4da846055cf36f8821` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:54.576003 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170e9db-dc8d-4b44-81f3-3084f2343789 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6db951fe182a4a4da846055cf36f8821)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:54.578385 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170e9db-dc8d-4b44-81f3-3084f2343789 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:54.605352 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:54.817766 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0170e9db-dc8d-4b44-81f3-3084f2343789 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6db951fe182a4a4da846055cf36f8821'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:54.878938 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0170e9db-dc8d-4b44-81f3-3084f2343789 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6db951fe182a4a4da846055cf36f8821'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:54.884147 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0170e9db-dc8d-4b44-81f3-3084f2343789 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6db951fe182a4a4da846055cf36f8821'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:54.884147 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0170e9db-dc8d-4b44-81f3-3084f2343789 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6db951fe182a4a4da846055cf36f8821'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:54.894139 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-0170e9db-dc8d-4b44-81f3-3084f2343789 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6db951fe182a4a4da846055cf36f8821'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:54.931456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1393/2788] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:49:54 2022] DELETE /identity/v3/users/6db951fe182a4a4da846055cf36f8821 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:49:54.936039 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ab41b-6734-4779-8825-f142acde7b97 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:54.936476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ab41b-6734-4779-8825-f142acde7b97 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:54.936694 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ab41b-6734-4779-8825-f142acde7b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:54.958855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-ad9ab41b-6734-4779-8825-f142acde7b97 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:49:54.960137 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ad9ab41b-6734-4779-8825-f142acde7b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:55.038597 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1396/2789] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:49:55.048217 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170e9db-dc8d-4b44-81f3-3084f2343789 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:55.049713 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-0170e9db-dc8d-4b44-81f3-3084f2343789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:55.132466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170e9db-dc8d-4b44-81f3-3084f2343789 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:55.137662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2a77b-cc49-451a-a6cf-267ed344e4d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:55.138654 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2a77b-cc49-451a-a6cf-267ed344e4d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:55.139148 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2a77b-cc49-451a-a6cf-267ed344e4d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:55.139564 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2a77b-cc49-451a-a6cf-267ed344e4d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:55.155959 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2a77b-cc49-451a-a6cf-267ed344e4d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:55.157769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2a77b-cc49-451a-a6cf-267ed344e4d8 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:55.179733 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1394/2790] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:49:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:49:55.711120 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ab41b-6734-4779-8825-f142acde7b97 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:49:55.717287 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ad9ab41b-6734-4779-8825-f142acde7b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:49:55.732979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ab41b-6734-4779-8825-f142acde7b97 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:49:55.738066 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830ff03-736a-4c3f-9c61-b9709d160ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:49:55.739181 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830ff03-736a-4c3f-9c61-b9709d160ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:49:55.739548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830ff03-736a-4c3f-9c61-b9709d160ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:49:55.739941 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830ff03-736a-4c3f-9c61-b9709d160ba9 admin admin] PATH_INFO: `/v3/projects/615dd52181464f35a8e6d2247c3668f7` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:49:55.741652 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830ff03-736a-4c3f-9c61-b9709d160ba9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=615dd52181464f35a8e6d2247c3668f7)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:49:55.746993 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830ff03-736a-4c3f-9c61-b9709d160ba9 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:49:55.908443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e830ff03-736a-4c3f-9c61-b9709d160ba9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '615dd52181464f35a8e6d2247c3668f7', 'request_id': 'req-e830ff03-736a-4c3f-9c61-b9709d160ba9'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:55.912199 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:49:55.916251 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e830ff03-736a-4c3f-9c61-b9709d160ba9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '615dd52181464f35a8e6d2247c3668f7', 'request_id': 'req-e830ff03-736a-4c3f-9c61-b9709d160ba9'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:55.923786 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e830ff03-736a-4c3f-9c61-b9709d160ba9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '615dd52181464f35a8e6d2247c3668f7', 'request_id': 'req-e830ff03-736a-4c3f-9c61-b9709d160ba9'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:55.995516 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e830ff03-736a-4c3f-9c61-b9709d160ba9 admin admin] The token cache is being invalidate because project 615dd52181464f35a8e6d2247c3668f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:49:55.996860 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e830ff03-736a-4c3f-9c61-b9709d160ba9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:55.997013 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-e830ff03-736a-4c3f-9c61-b9709d160ba9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:49:55.999062 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1397/2791] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:49:55 2022] DELETE /identity/v3/projects/615dd52181464f35a8e6d2247c3668f7 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:50:00.453215 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575edb1-57c2-41c3-991f-6e6fb34daccd None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:50:00.453617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575edb1-57c2-41c3-991f-6e6fb34daccd None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:50:00.453801 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575edb1-57c2-41c3-991f-6e6fb34daccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:50:00.471626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-3575edb1-57c2-41c3-991f-6e6fb34daccd None None] MFA Rules not processed for user `745d930ebbab403c9e974f47d3719035`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:50:00.472835 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3575edb1-57c2-41c3-991f-6e6fb34daccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:50:00.527588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1395/2792] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:50:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:50:00.783722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830ff03-736a-4c3f-9c61-b9709d160ba9 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:50:00.785240 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e830ff03-736a-4c3f-9c61-b9709d160ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:50:00.840577 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830ff03-736a-4c3f-9c61-b9709d160ba9 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:50:00.844231 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f7590-1c1f-4ef6-95ff-984c547e7106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:50:00.844876 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f7590-1c1f-4ef6-95ff-984c547e7106 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:50:00.845158 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f7590-1c1f-4ef6-95ff-984c547e7106 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:50:00.845431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f7590-1c1f-4ef6-95ff-984c547e7106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:50:00.847227 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-211f7590-1c1f-4ef6-95ff-984c547e7106 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:50:00.856794 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f7590-1c1f-4ef6-95ff-984c547e7106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:50:00.858208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f7590-1c1f-4ef6-95ff-984c547e7106 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:50:00.866127 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1398/2793] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:50:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:50:01.255864 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575edb1-57c2-41c3-991f-6e6fb34daccd None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:50:01.257285 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3575edb1-57c2-41c3-991f-6e6fb34daccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:50:01.314597 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:50:01.338695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575edb1-57c2-41c3-991f-6e6fb34daccd None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:50:01.343819 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a450e-93a6-4cde-88f5-aec59036bf91 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a30cee766814c8685578819261705ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:50:01.344441 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a450e-93a6-4cde-88f5-aec59036bf91 service placement] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:50:01.344722 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a450e-93a6-4cde-88f5-aec59036bf91 service placement] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:50:01.344994 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a450e-93a6-4cde-88f5-aec59036bf91 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:50:01.346718 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5d3a450e-93a6-4cde-88f5-aec59036bf91 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:50:01.357933 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a450e-93a6-4cde-88f5-aec59036bf91 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:50:01.359522 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a450e-93a6-4cde-88f5-aec59036bf91 service placement] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:50:01.369175 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1396/2794] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:50:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:50:03.891587 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a1099-f798-427e-91a2-1139025adf56 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:50:03.892099 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a1099-f798-427e-91a2-1139025adf56 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:50:03.892267 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a1099-f798-427e-91a2-1139025adf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:50:03.912317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-c18a1099-f798-427e-91a2-1139025adf56 None None] MFA Rules not processed for user `a4f499707998403d8f833eb783599466`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:50:03.913561 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c18a1099-f798-427e-91a2-1139025adf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:50:03.971438 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1399/2795] 10.222.0.27 () {58 vars in 1009 bytes} [Sat Jul 16 15:50:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:50:03.984094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a450e-93a6-4cde-88f5-aec59036bf91 service placement] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:50:03.985524 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5d3a450e-93a6-4cde-88f5-aec59036bf91 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:50:04.054794 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a450e-93a6-4cde-88f5-aec59036bf91 service placement] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:50:04.060213 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bcc7f5-398d-40d0-b491-2a42d448e0e3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1464c83fb294a3d90e5763ef648e834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:50:04.060865 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcc7f5-398d-40d0-b491-2a42d448e0e3 service swift] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:50:04.061148 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcc7f5-398d-40d0-b491-2a42d448e0e3 service swift] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:50:04.061444 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcc7f5-398d-40d0-b491-2a42d448e0e3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:50:04.070218 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bcc7f5-398d-40d0-b491-2a42d448e0e3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:50:04.075754 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bcc7f5-398d-40d0-b491-2a42d448e0e3 service swift] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:50:04.081040 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1397/2796] 10.222.0.27 () {60 vars in 1384 bytes} [Sat Jul 16 15:50:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 15:50:05.362752 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a1099-f798-427e-91a2-1139025adf56 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:50:05.364912 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-c18a1099-f798-427e-91a2-1139025adf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:50:05.498508 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a1099-f798-427e-91a2-1139025adf56 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:50:05.508675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a8836-5556-438a-8d46-f9f02184c428 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:50:05.509345 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a8836-5556-438a-8d46-f9f02184c428 service nova] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:50:05.509755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a8836-5556-438a-8d46-f9f02184c428 service nova] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:50:05.510037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a8836-5556-438a-8d46-f9f02184c428 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:50:05.512096 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b89a8836-5556-438a-8d46-f9f02184c428 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:50:05.530680 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a8836-5556-438a-8d46-f9f02184c428 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:50:05.532570 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a8836-5556-438a-8d46-f9f02184c428 service nova] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:50:05.548343 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1400/2797] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:50:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:50:14.077523 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=45 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:50:15.541384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:50:18.127577 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bcc7f5-398d-40d0-b491-2a42d448e0e3 service swift] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:50:18.139726 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bcc7f5-398d-40d0-b491-2a42d448e0e3 service swift] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:50:18.145230 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c26a5-a5fd-4fd7-883d-a379866b19bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:50:18.145804 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c26a5-a5fd-4fd7-883d-a379866b19bb service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:50:18.146083 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c26a5-a5fd-4fd7-883d-a379866b19bb service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:50:18.146354 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c26a5-a5fd-4fd7-883d-a379866b19bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:50:18.148113 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-562c26a5-a5fd-4fd7-883d-a379866b19bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:50:18.161153 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c26a5-a5fd-4fd7-883d-a379866b19bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:50:18.162716 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c26a5-a5fd-4fd7-883d-a379866b19bb service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:50:18.173917 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1398/2798] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:50:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:50:28.168734 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=6 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:50:31.226974 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a8836-5556-438a-8d46-f9f02184c428 service nova] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:50:31.236614 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a8836-5556-438a-8d46-f9f02184c428 service nova] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:50:31.241234 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ebc6d-b7d4-4f5c-a90e-6d3a1d79f4c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:50:31.241234 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ebc6d-b7d4-4f5c-a90e-6d3a1d79f4c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:50:31.241702 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ebc6d-b7d4-4f5c-a90e-6d3a1d79f4c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:50:31.241985 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ebc6d-b7d4-4f5c-a90e-6d3a1d79f4c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:50:31.243781 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-105ebc6d-b7d4-4f5c-a90e-6d3a1d79f4c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:50:31.309727 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ebc6d-b7d4-4f5c-a90e-6d3a1d79f4c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:50:31.311699 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ebc6d-b7d4-4f5c-a90e-6d3a1d79f4c4 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:50:31.321505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1401/2799] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:50:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:50:41.317093 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=30 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:09.251478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c26a5-a5fd-4fd7-883d-a379866b19bb service nova] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:09.252909 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-562c26a5-a5fd-4fd7-883d-a379866b19bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:09.264430 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c26a5-a5fd-4fd7-883d-a379866b19bb service nova] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:09.267944 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209087ae-0a05-4f57-94da-9913ffefa345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:09.269169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209087ae-0a05-4f57-94da-9913ffefa345 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:09.269169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209087ae-0a05-4f57-94da-9913ffefa345 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:09.269169 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209087ae-0a05-4f57-94da-9913ffefa345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:09.270818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-209087ae-0a05-4f57-94da-9913ffefa345 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:09.333421 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209087ae-0a05-4f57-94da-9913ffefa345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:09.334925 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209087ae-0a05-4f57-94da-9913ffefa345 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:09.345227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1399/2800] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:51:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:51:12.998533 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ebc6d-b7d4-4f5c-a90e-6d3a1d79f4c4 service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:13.004745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-105ebc6d-b7d4-4f5c-a90e-6d3a1d79f4c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:13.020780 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ebc6d-b7d4-4f5c-a90e-6d3a1d79f4c4 service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:13.025658 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a3874-8a65-4295-9835-23fe11b91b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:13.026676 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a3874-8a65-4295-9835-23fe11b91b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:13.027020 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a3874-8a65-4295-9835-23fe11b91b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:13.027298 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a3874-8a65-4295-9835-23fe11b91b45 admin admin] PATH_INFO: `/v3/users/2517f9ee6c3b4d52b44d5da510e7cbf8` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:13.028483 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a3874-8a65-4295-9835-23fe11b91b45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2517f9ee6c3b4d52b44d5da510e7cbf8)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:13.031140 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a3874-8a65-4295-9835-23fe11b91b45 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:13.230034 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-6d0a3874-8a65-4295-9835-23fe11b91b45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2517f9ee6c3b4d52b44d5da510e7cbf8'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:13.294851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-6d0a3874-8a65-4295-9835-23fe11b91b45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2517f9ee6c3b4d52b44d5da510e7cbf8'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:13.302652 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-6d0a3874-8a65-4295-9835-23fe11b91b45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2517f9ee6c3b4d52b44d5da510e7cbf8'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:13.315109 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-6d0a3874-8a65-4295-9835-23fe11b91b45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2517f9ee6c3b4d52b44d5da510e7cbf8'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:13.315751 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-6d0a3874-8a65-4295-9835-23fe11b91b45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2517f9ee6c3b4d52b44d5da510e7cbf8'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:13.321541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1402/2801] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:51:12 2022] DELETE /identity/v3/users/2517f9ee6c3b4d52b44d5da510e7cbf8 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:13.605313 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209087ae-0a05-4f57-94da-9913ffefa345 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:13.616477 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209087ae-0a05-4f57-94da-9913ffefa345 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:13.674467 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9f2c0-7b57-4425-ad5b-3001c1aee1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:13.675472 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9f2c0-7b57-4425-ad5b-3001c1aee1dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:13.675756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9f2c0-7b57-4425-ad5b-3001c1aee1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:13.676034 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9f2c0-7b57-4425-ad5b-3001c1aee1dd admin admin] PATH_INFO: `/v3/projects/e7ad41a7c9214f44a07aba8b9f86c6f4` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:13.677262 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9f2c0-7b57-4425-ad5b-3001c1aee1dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7ad41a7c9214f44a07aba8b9f86c6f4)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:13.679201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9f2c0-7b57-4425-ad5b-3001c1aee1dd admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:13.798755 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b7c9f2c0-7b57-4425-ad5b-3001c1aee1dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7ad41a7c9214f44a07aba8b9f86c6f4', 'request_id': 'req-b7c9f2c0-7b57-4425-ad5b-3001c1aee1dd'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:13.804216 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b7c9f2c0-7b57-4425-ad5b-3001c1aee1dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7ad41a7c9214f44a07aba8b9f86c6f4', 'request_id': 'req-b7c9f2c0-7b57-4425-ad5b-3001c1aee1dd'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:13.810277 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b7c9f2c0-7b57-4425-ad5b-3001c1aee1dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7ad41a7c9214f44a07aba8b9f86c6f4', 'request_id': 'req-b7c9f2c0-7b57-4425-ad5b-3001c1aee1dd'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:13.894879 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b7c9f2c0-7b57-4425-ad5b-3001c1aee1dd admin admin] The token cache is being invalidate because project e7ad41a7c9214f44a07aba8b9f86c6f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80256) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:51:13.895197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b7c9f2c0-7b57-4425-ad5b-3001c1aee1dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:13.895891 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b7c9f2c0-7b57-4425-ad5b-3001c1aee1dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:13.897433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1400/2802] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:51:13 2022] DELETE /identity/v3/projects/e7ad41a7c9214f44a07aba8b9f86c6f4 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:13.905633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aed6e9-a575-487e-831f-a2f3859c242f None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:13.905633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aed6e9-a575-487e-831f-a2f3859c242f None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:13.905633 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aed6e9-a575-487e-831f-a2f3859c242f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:13.923913 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-d4aed6e9-a575-487e-831f-a2f3859c242f None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:13.925095 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-d4aed6e9-a575-487e-831f-a2f3859c242f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:13.985123 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1403/2803] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:13.993093 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945114e-db3f-4481-9397-d81993de3957 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:13.993384 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945114e-db3f-4481-9397-d81993de3957 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:13.993660 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945114e-db3f-4481-9397-d81993de3957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:14.011707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-a945114e-db3f-4481-9397-d81993de3957 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:14.012908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-a945114e-db3f-4481-9397-d81993de3957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:14.020745 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1401/2804] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:14.024929 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aed6e9-a575-487e-831f-a2f3859c242f None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:14.035387 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aed6e9-a575-487e-831f-a2f3859c242f None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:14.039861 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe65f6-aef5-4539-94c0-1e4da9a5b278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:14.040494 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe65f6-aef5-4539-94c0-1e4da9a5b278 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:14.040772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe65f6-aef5-4539-94c0-1e4da9a5b278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:14.041041 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe65f6-aef5-4539-94c0-1e4da9a5b278 admin admin] PATH_INFO: `/v3/domains` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:14.041756 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe65f6-aef5-4539-94c0-1e4da9a5b278 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80257) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:51:14.042053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe65f6-aef5-4539-94c0-1e4da9a5b278 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:14.043663 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe65f6-aef5-4539-94c0-1e4da9a5b278 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:14.055546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1404/2805] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 15:51:14.059769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945114e-db3f-4481-9397-d81993de3957 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:14.068999 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945114e-db3f-4481-9397-d81993de3957 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:14.073672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1dc72-38ea-4278-ab82-1305d7d4804a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:14.074317 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1dc72-38ea-4278-ab82-1305d7d4804a admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:14.074599 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1dc72-38ea-4278-ab82-1305d7d4804a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:14.074880 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1dc72-38ea-4278-ab82-1305d7d4804a admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:14.075601 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1dc72-38ea-4278-ab82-1305d7d4804a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:14.077585 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1dc72-38ea-4278-ab82-1305d7d4804a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:14.091702 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:14.185023 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1402/2806] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:51:14 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:51:14.189680 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe65f6-aef5-4539-94c0-1e4da9a5b278 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:14.199649 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe65f6-aef5-4539-94c0-1e4da9a5b278 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:14.255983 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36305d51-42e4-49c2-aba4-93106e3920f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:14.256646 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36305d51-42e4-49c2-aba4-93106e3920f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:14.256932 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36305d51-42e4-49c2-aba4-93106e3920f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:14.257212 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36305d51-42e4-49c2-aba4-93106e3920f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:14.257887 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36305d51-42e4-49c2-aba4-93106e3920f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:14.259838 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36305d51-42e4-49c2-aba4-93106e3920f2 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:14.266655 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:14.358031 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1405/2807] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:51:14 2022] POST /identity/v3/users => generated 290 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:51:14.362134 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1dc72-38ea-4278-ab82-1305d7d4804a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:14.372481 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1dc72-38ea-4278-ab82-1305d7d4804a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:14.377372 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fd134-d179-40a7-b227-f010174f8c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:14.378029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fd134-d179-40a7-b227-f010174f8c47 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:14.378317 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fd134-d179-40a7-b227-f010174f8c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:14.378597 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04fd134-d179-40a7-b227-f010174f8c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:14.379360 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04fd134-d179-40a7-b227-f010174f8c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:14.380957 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04fd134-d179-40a7-b227-f010174f8c47 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:14.392141 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1403/2808] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:14.395961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36305d51-42e4-49c2-aba4-93106e3920f2 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:14.406611 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36305d51-42e4-49c2-aba4-93106e3920f2 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:14.411433 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab89358-d1c7-48b8-b22c-6f0029883404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:14.412146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab89358-d1c7-48b8-b22c-6f0029883404 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:14.412423 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab89358-d1c7-48b8-b22c-6f0029883404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:14.412692 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab89358-d1c7-48b8-b22c-6f0029883404 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:14.413410 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab89358-d1c7-48b8-b22c-6f0029883404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:14.415043 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab89358-d1c7-48b8-b22c-6f0029883404 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:14.425983 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1406/2809] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:14.430029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fd134-d179-40a7-b227-f010174f8c47 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:14.440612 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04fd134-d179-40a7-b227-f010174f8c47 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:14.444671 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94341d0a-2819-4c75-a754-deb0af3b06a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:14.445550 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94341d0a-2819-4c75-a754-deb0af3b06a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:14.445827 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94341d0a-2819-4c75-a754-deb0af3b06a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:14.446101 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94341d0a-2819-4c75-a754-deb0af3b06a1 admin admin] PATH_INFO: `/v3/projects/2ca7d5431613411cae822f1e093c8cbc/users/fbdd8edaeba04715bb11851df388b2e4/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:14.461999 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94341d0a-2819-4c75-a754-deb0af3b06a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ca7d5431613411cae822f1e093c8cbc, user_id=fbdd8edaeba04715bb11851df388b2e4, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:14.464716 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94341d0a-2819-4c75-a754-deb0af3b06a1 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:14.550466 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1404/2810] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:51:14 2022] PUT /identity/v3/projects/2ca7d5431613411cae822f1e093c8cbc/users/fbdd8edaeba04715bb11851df388b2e4/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:51:14.560823 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b415ac-81f9-42c6-8f15-b246da9fb933 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:14.561167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b415ac-81f9-42c6-8f15-b246da9fb933 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:14.561448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b415ac-81f9-42c6-8f15-b246da9fb933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:14.588296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-84b415ac-81f9-42c6-8f15-b246da9fb933 None None] MFA Rules not processed for user `fbdd8edaeba04715bb11851df388b2e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:14.589507 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-84b415ac-81f9-42c6-8f15-b246da9fb933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:14.654328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1407/2811] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:14.663484 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94341d0a-2819-4c75-a754-deb0af3b06a1 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:14.725438 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94341d0a-2819-4c75-a754-deb0af3b06a1 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:14.730052 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71b37e-168a-4b2b-a3d0-91d36da4502b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:14.730706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71b37e-168a-4b2b-a3d0-91d36da4502b admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:14.731024 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71b37e-168a-4b2b-a3d0-91d36da4502b admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:14.731302 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71b37e-168a-4b2b-a3d0-91d36da4502b admin admin] PATH_INFO: `/v3/projects` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:14.731993 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db71b37e-168a-4b2b-a3d0-91d36da4502b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:14.733934 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db71b37e-168a-4b2b-a3d0-91d36da4502b admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:14.810342 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1405/2812] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:51:14 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:51:14.815077 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b415ac-81f9-42c6-8f15-b246da9fb933 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:14.825741 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b415ac-81f9-42c6-8f15-b246da9fb933 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:14.881389 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd08f2-53d3-446b-b92d-de95c64a9c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:14.882056 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd08f2-53d3-446b-b92d-de95c64a9c3f admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:14.882335 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd08f2-53d3-446b-b92d-de95c64a9c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:14.882609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd08f2-53d3-446b-b92d-de95c64a9c3f admin admin] PATH_INFO: `/v3/users` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:14.883366 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd08f2-53d3-446b-b92d-de95c64a9c3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:14.885271 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd08f2-53d3-446b-b92d-de95c64a9c3f admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:14.998978 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1408/2813] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:51:14 2022] POST /identity/v3/users => generated 296 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:51:15.003310 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71b37e-168a-4b2b-a3d0-91d36da4502b admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:15.013577 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71b37e-168a-4b2b-a3d0-91d36da4502b admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:15.018941 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e404e-7886-48ef-be65-b15bae97ec8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:15.019603 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e404e-7886-48ef-be65-b15bae97ec8f admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:15.019887 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e404e-7886-48ef-be65-b15bae97ec8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:15.020167 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e404e-7886-48ef-be65-b15bae97ec8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:15.020890 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e404e-7886-48ef-be65-b15bae97ec8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:15.022478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e404e-7886-48ef-be65-b15bae97ec8f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:15.033319 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1406/2814] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:15.037390 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd08f2-53d3-446b-b92d-de95c64a9c3f admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:15.047866 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd08f2-53d3-446b-b92d-de95c64a9c3f admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:15.052443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e157e-599e-4d87-a603-845ecf1d535f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:15.053331 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e157e-599e-4d87-a603-845ecf1d535f admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:15.053597 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e157e-599e-4d87-a603-845ecf1d535f admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:15.053897 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e157e-599e-4d87-a603-845ecf1d535f admin admin] PATH_INFO: `/v3/projects/37677b82eb9a4163af6c0b9d92976e7f/users/7400e051f7244edb80697a95c0150738/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:15.068809 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e157e-599e-4d87-a603-845ecf1d535f admin admin] RBAC: Authorizing `identity:create_grant(project_id=37677b82eb9a4163af6c0b9d92976e7f, user_id=7400e051f7244edb80697a95c0150738, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:15.071457 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e157e-599e-4d87-a603-845ecf1d535f admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:15.195382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1409/2815] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:51:15 2022] PUT /identity/v3/projects/37677b82eb9a4163af6c0b9d92976e7f/users/7400e051f7244edb80697a95c0150738/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:15.199270 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e404e-7886-48ef-be65-b15bae97ec8f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:15.209488 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e404e-7886-48ef-be65-b15bae97ec8f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:15.266765 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c01f5-bdde-4381-9cee-1d99cd9969fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:15.267376 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c01f5-bdde-4381-9cee-1d99cd9969fb admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:15.267654 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c01f5-bdde-4381-9cee-1d99cd9969fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:15.267922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c01f5-bdde-4381-9cee-1d99cd9969fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:15.268662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c01f5-bdde-4381-9cee-1d99cd9969fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:15.270236 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c01f5-bdde-4381-9cee-1d99cd9969fb admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:15.282070 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1407/2816] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:15.286186 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e157e-599e-4d87-a603-845ecf1d535f admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:15.297176 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e157e-599e-4d87-a603-845ecf1d535f admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:15.301717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858f755-b0f8-4bb7-bc81-bf24c8824792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:15.302586 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858f755-b0f8-4bb7-bc81-bf24c8824792 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:15.302859 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858f755-b0f8-4bb7-bc81-bf24c8824792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:15.303174 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858f755-b0f8-4bb7-bc81-bf24c8824792 admin admin] PATH_INFO: `/v3/projects/37677b82eb9a4163af6c0b9d92976e7f/users/7400e051f7244edb80697a95c0150738/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:15.305124 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8858f755-b0f8-4bb7-bc81-bf24c8824792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37677b82eb9a4163af6c0b9d92976e7f, user_id=7400e051f7244edb80697a95c0150738, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:15.307760 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8858f755-b0f8-4bb7-bc81-bf24c8824792 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:15.318642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1410/2817] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:51:15 2022] PUT /identity/v3/projects/37677b82eb9a4163af6c0b9d92976e7f/users/7400e051f7244edb80697a95c0150738/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:15.329767 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2d736-b4b7-4148-8681-801d12b58469 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:15.330092 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2d736-b4b7-4148-8681-801d12b58469 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:15.330379 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2d736-b4b7-4148-8681-801d12b58469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:15.356533 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-42c2d736-b4b7-4148-8681-801d12b58469 None None] MFA Rules not processed for user `7400e051f7244edb80697a95c0150738`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:15.357721 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-42c2d736-b4b7-4148-8681-801d12b58469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:15.377846 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:15.427378 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1408/2818] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:15.440515 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858f755-b0f8-4bb7-bc81-bf24c8824792 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:15.442183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-8858f755-b0f8-4bb7-bc81-bf24c8824792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:15.464396 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:15.509590 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858f755-b0f8-4bb7-bc81-bf24c8824792 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:15.513262 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3edd2-8df7-420f-889c-4f145fc033b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:15.513899 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3edd2-8df7-420f-889c-4f145fc033b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:15.514407 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3edd2-8df7-420f-889c-4f145fc033b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:15.514699 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3edd2-8df7-420f-889c-4f145fc033b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:15.524224 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3edd2-8df7-420f-889c-4f145fc033b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:15.525881 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3edd2-8df7-420f-889c-4f145fc033b9 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:15.535996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1411/2819] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:51:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:51:16.355500 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2d736-b4b7-4148-8681-801d12b58469 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:16.364957 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2d736-b4b7-4148-8681-801d12b58469 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:16.369632 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c6cee-9bd7-4f0d-80d9-796fc441ecd6 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7400e051f7244edb80697a95c0150738', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37677b82eb9a4163af6c0b9d92976e7f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:16.370295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c6cee-9bd7-4f0d-80d9-796fc441ecd6 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:16.370590 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c6cee-9bd7-4f0d-80d9-796fc441ecd6 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:16.370868 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c6cee-9bd7-4f0d-80d9-796fc441ecd6 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:16.371658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c6cee-9bd7-4f0d-80d9-796fc441ecd6 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:51:16.371969 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c6cee-9bd7-4f0d-80d9-796fc441ecd6 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:16.373623 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c6cee-9bd7-4f0d-80d9-796fc441ecd6 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:16.385260 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1409/2820] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 15:51:16.389162 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3edd2-8df7-420f-889c-4f145fc033b9 service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:16.398677 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3edd2-8df7-420f-889c-4f145fc033b9 service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:16.403755 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b47a12-0d14-4ea4-b5ca-13074d8b5840 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7400e051f7244edb80697a95c0150738', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37677b82eb9a4163af6c0b9d92976e7f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:16.404409 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b47a12-0d14-4ea4-b5ca-13074d8b5840 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:16.404698 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b47a12-0d14-4ea4-b5ca-13074d8b5840 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:16.404990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b47a12-0d14-4ea4-b5ca-13074d8b5840 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:16.406311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b47a12-0d14-4ea4-b5ca-13074d8b5840 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:16.408267 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b47a12-0d14-4ea4-b5ca-13074d8b5840 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:16.490674 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1412/2821] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:51:16 2022] POST /identity/v3/projects => generated 354 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:51:16.494760 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c6cee-9bd7-4f0d-80d9-796fc441ecd6 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:16.505625 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c6cee-9bd7-4f0d-80d9-796fc441ecd6 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:16.561804 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3da6e9-9463-4ddc-af33-c00641840bb8 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7400e051f7244edb80697a95c0150738', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37677b82eb9a4163af6c0b9d92976e7f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:16.562520 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3da6e9-9463-4ddc-af33-c00641840bb8 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:16.562809 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3da6e9-9463-4ddc-af33-c00641840bb8 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:16.563177 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3da6e9-9463-4ddc-af33-c00641840bb8 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:16.563910 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3da6e9-9463-4ddc-af33-c00641840bb8 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:51:16.564224 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3da6e9-9463-4ddc-af33-c00641840bb8 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:16.565819 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3da6e9-9463-4ddc-af33-c00641840bb8 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:16.576986 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1410/2822] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:51:16.612850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b47a12-0d14-4ea4-b5ca-13074d8b5840 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:16.623161 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b47a12-0d14-4ea4-b5ca-13074d8b5840 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:16.628354 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da19d28-a94d-46d9-b5c9-95ab6745b448 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7400e051f7244edb80697a95c0150738', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37677b82eb9a4163af6c0b9d92976e7f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:16.629204 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da19d28-a94d-46d9-b5c9-95ab6745b448 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:16.629489 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da19d28-a94d-46d9-b5c9-95ab6745b448 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:16.629765 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da19d28-a94d-46d9-b5c9-95ab6745b448 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] PATH_INFO: `/v3/projects/700d36e483c640d797f9525caa1e1bc9` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:16.631001 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da19d28-a94d-46d9-b5c9-95ab6745b448 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] RBAC: Authorizing `identity:delete_project(project_id=700d36e483c640d797f9525caa1e1bc9)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:16.632902 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da19d28-a94d-46d9-b5c9-95ab6745b448 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:16.784492 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-2da19d28-a94d-46d9-b5c9-95ab6745b448 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '700d36e483c640d797f9525caa1e1bc9', 'request_id': 'req-2da19d28-a94d-46d9-b5c9-95ab6745b448'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:16.789939 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-2da19d28-a94d-46d9-b5c9-95ab6745b448 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '700d36e483c640d797f9525caa1e1bc9', 'request_id': 'req-2da19d28-a94d-46d9-b5c9-95ab6745b448'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:16.796734 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-2da19d28-a94d-46d9-b5c9-95ab6745b448 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '700d36e483c640d797f9525caa1e1bc9', 'request_id': 'req-2da19d28-a94d-46d9-b5c9-95ab6745b448'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:16.875037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-2da19d28-a94d-46d9-b5c9-95ab6745b448 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] The token cache is being invalidate because project 700d36e483c640d797f9525caa1e1bc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:51:16.875329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-2da19d28-a94d-46d9-b5c9-95ab6745b448 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:16.875961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-2da19d28-a94d-46d9-b5c9-95ab6745b448 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:16.877512 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1413/2823] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:51:16 2022] DELETE /identity/v3/projects/700d36e483c640d797f9525caa1e1bc9 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:17.630605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3da6e9-9463-4ddc-af33-c00641840bb8 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:17.632496 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5b3da6e9-9463-4ddc-af33-c00641840bb8 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:17.697248 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3da6e9-9463-4ddc-af33-c00641840bb8 tempest-VolumeQuotaClassesTest-138610791 tempest-VolumeQuotaClassesTest-138610791-project-admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:17.739048 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=100 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:17.753340 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d476a04-a539-4206-8d21-02891d91e6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:17.754161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d476a04-a539-4206-8d21-02891d91e6ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:17.754444 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d476a04-a539-4206-8d21-02891d91e6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:17.754723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d476a04-a539-4206-8d21-02891d91e6ab admin admin] PATH_INFO: `/v3/users/fbdd8edaeba04715bb11851df388b2e4` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:17.756032 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d476a04-a539-4206-8d21-02891d91e6ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbdd8edaeba04715bb11851df388b2e4)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:17.757943 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d476a04-a539-4206-8d21-02891d91e6ab admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:17.966352 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9d476a04-a539-4206-8d21-02891d91e6ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbdd8edaeba04715bb11851df388b2e4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:18.037670 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9d476a04-a539-4206-8d21-02891d91e6ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbdd8edaeba04715bb11851df388b2e4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:18.038213 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9d476a04-a539-4206-8d21-02891d91e6ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbdd8edaeba04715bb11851df388b2e4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:18.038814 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9d476a04-a539-4206-8d21-02891d91e6ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbdd8edaeba04715bb11851df388b2e4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:18.046105 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-9d476a04-a539-4206-8d21-02891d91e6ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbdd8edaeba04715bb11851df388b2e4'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:18.063865 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1411/2824] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:51:17 2022] DELETE /identity/v3/users/fbdd8edaeba04715bb11851df388b2e4 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:18.068349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9179a-2af1-4a65-aedc-ee270de6afc2 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:18.068821 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9179a-2af1-4a65-aedc-ee270de6afc2 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:18.068984 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9179a-2af1-4a65-aedc-ee270de6afc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:18.086246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-99e9179a-2af1-4a65-aedc-ee270de6afc2 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:18.087471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-99e9179a-2af1-4a65-aedc-ee270de6afc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:18.146418 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1414/2825] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:18.156463 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d476a04-a539-4206-8d21-02891d91e6ab admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:18.157876 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-9d476a04-a539-4206-8d21-02891d91e6ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:18.214251 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d476a04-a539-4206-8d21-02891d91e6ab admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:18.217824 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a996c64-d957-4da4-a258-3ca8b660dc53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:18.218462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a996c64-d957-4da4-a258-3ca8b660dc53 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:18.218767 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a996c64-d957-4da4-a258-3ca8b660dc53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:18.219076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a996c64-d957-4da4-a258-3ca8b660dc53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:18.228054 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a996c64-d957-4da4-a258-3ca8b660dc53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:18.229478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a996c64-d957-4da4-a258-3ca8b660dc53 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:18.239177 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1412/2826] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:51:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:51:18.611452 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9179a-2af1-4a65-aedc-ee270de6afc2 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:18.612996 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-99e9179a-2af1-4a65-aedc-ee270de6afc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:18.626100 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9179a-2af1-4a65-aedc-ee270de6afc2 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:18.630873 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195c1f6-7f32-4d3e-89e4-ce5727f50f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:18.631752 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195c1f6-7f32-4d3e-89e4-ce5727f50f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:18.632036 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195c1f6-7f32-4d3e-89e4-ce5727f50f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:18.632320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195c1f6-7f32-4d3e-89e4-ce5727f50f18 admin admin] PATH_INFO: `/v3/projects/2ca7d5431613411cae822f1e093c8cbc` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:18.633508 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195c1f6-7f32-4d3e-89e4-ce5727f50f18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ca7d5431613411cae822f1e093c8cbc)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:18.635455 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195c1f6-7f32-4d3e-89e4-ce5727f50f18 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:18.646657 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:18.780987 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-3195c1f6-7f32-4d3e-89e4-ce5727f50f18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ca7d5431613411cae822f1e093c8cbc', 'request_id': 'req-3195c1f6-7f32-4d3e-89e4-ce5727f50f18'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:18.786279 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-3195c1f6-7f32-4d3e-89e4-ce5727f50f18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ca7d5431613411cae822f1e093c8cbc', 'request_id': 'req-3195c1f6-7f32-4d3e-89e4-ce5727f50f18'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:18.792425 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-3195c1f6-7f32-4d3e-89e4-ce5727f50f18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ca7d5431613411cae822f1e093c8cbc', 'request_id': 'req-3195c1f6-7f32-4d3e-89e4-ce5727f50f18'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:18.865812 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-3195c1f6-7f32-4d3e-89e4-ce5727f50f18 admin admin] The token cache is being invalidate because project 2ca7d5431613411cae822f1e093c8cbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:51:18.866028 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-3195c1f6-7f32-4d3e-89e4-ce5727f50f18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:18.866673 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-3195c1f6-7f32-4d3e-89e4-ce5727f50f18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:18.868212 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1415/2827] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:51:18 2022] DELETE /identity/v3/projects/2ca7d5431613411cae822f1e093c8cbc => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:51:18.872162 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a996c64-d957-4da4-a258-3ca8b660dc53 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:18.882049 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a996c64-d957-4da4-a258-3ca8b660dc53 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:18.938395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb9e27-3dc5-4f71-8aff-f9fc59862529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:18.941019 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb9e27-3dc5-4f71-8aff-f9fc59862529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:18.941019 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb9e27-3dc5-4f71-8aff-f9fc59862529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:18.941019 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb9e27-3dc5-4f71-8aff-f9fc59862529 admin admin] PATH_INFO: `/v3/users/7400e051f7244edb80697a95c0150738` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:18.941279 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb9e27-3dc5-4f71-8aff-f9fc59862529 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7400e051f7244edb80697a95c0150738)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:18.943071 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb9e27-3dc5-4f71-8aff-f9fc59862529 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:18.948330 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:19.171471 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b1bb9e27-3dc5-4f71-8aff-f9fc59862529 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7400e051f7244edb80697a95c0150738'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:19.246109 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b1bb9e27-3dc5-4f71-8aff-f9fc59862529 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7400e051f7244edb80697a95c0150738'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:19.246720 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b1bb9e27-3dc5-4f71-8aff-f9fc59862529 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7400e051f7244edb80697a95c0150738'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:19.247339 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b1bb9e27-3dc5-4f71-8aff-f9fc59862529 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7400e051f7244edb80697a95c0150738'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:19.255084 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-b1bb9e27-3dc5-4f71-8aff-f9fc59862529 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7400e051f7244edb80697a95c0150738'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:19.273205 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1413/2828] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:51:18 2022] DELETE /identity/v3/users/7400e051f7244edb80697a95c0150738 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:19.725163 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195c1f6-7f32-4d3e-89e4-ce5727f50f18 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:19.726707 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3195c1f6-7f32-4d3e-89e4-ce5727f50f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:19.740166 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195c1f6-7f32-4d3e-89e4-ce5727f50f18 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:19.799396 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f9d4a-0673-46bf-93dc-a4f01ba5f4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:19.800260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f9d4a-0673-46bf-93dc-a4f01ba5f4fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:19.800542 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f9d4a-0673-46bf-93dc-a4f01ba5f4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:19.800816 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f9d4a-0673-46bf-93dc-a4f01ba5f4fc admin admin] PATH_INFO: `/v3/projects/37677b82eb9a4163af6c0b9d92976e7f` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:19.802005 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f9d4a-0673-46bf-93dc-a4f01ba5f4fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=37677b82eb9a4163af6c0b9d92976e7f)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:19.803970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f9d4a-0673-46bf-93dc-a4f01ba5f4fc admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:19.921746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-200f9d4a-0673-46bf-93dc-a4f01ba5f4fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37677b82eb9a4163af6c0b9d92976e7f', 'request_id': 'req-200f9d4a-0673-46bf-93dc-a4f01ba5f4fc'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:19.927317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-200f9d4a-0673-46bf-93dc-a4f01ba5f4fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37677b82eb9a4163af6c0b9d92976e7f', 'request_id': 'req-200f9d4a-0673-46bf-93dc-a4f01ba5f4fc'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:19.933450 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-200f9d4a-0673-46bf-93dc-a4f01ba5f4fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37677b82eb9a4163af6c0b9d92976e7f', 'request_id': 'req-200f9d4a-0673-46bf-93dc-a4f01ba5f4fc'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:20.014172 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-200f9d4a-0673-46bf-93dc-a4f01ba5f4fc admin admin] The token cache is being invalidate because project 37677b82eb9a4163af6c0b9d92976e7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:51:20.014389 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-200f9d4a-0673-46bf-93dc-a4f01ba5f4fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:20.015037 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-200f9d4a-0673-46bf-93dc-a4f01ba5f4fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:20.016688 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1416/2829] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:51:19 2022] DELETE /identity/v3/projects/37677b82eb9a4163af6c0b9d92976e7f => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:20.021506 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b04bf-5bdf-4564-8f98-5d568d421237 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:20.021978 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b04bf-5bdf-4564-8f98-5d568d421237 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:20.022075 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b04bf-5bdf-4564-8f98-5d568d421237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:20.040236 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-457b04bf-5bdf-4564-8f98-5d568d421237 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:20.041542 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-457b04bf-5bdf-4564-8f98-5d568d421237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:20.099168 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1414/2830] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:20.154740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2784686-56ff-4b7e-8bf7-928b02826a9e None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:20.155052 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2784686-56ff-4b7e-8bf7-928b02826a9e None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:20.155329 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2784686-56ff-4b7e-8bf7-928b02826a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:20.174076 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-b2784686-56ff-4b7e-8bf7-928b02826a9e None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:20.175296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b2784686-56ff-4b7e-8bf7-928b02826a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:20.183033 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1417/2831] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:20.187088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b04bf-5bdf-4564-8f98-5d568d421237 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:20.196804 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b04bf-5bdf-4564-8f98-5d568d421237 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:20.201323 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7425b-41ac-4dde-9b57-9b9edd9a9c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:20.201968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7425b-41ac-4dde-9b57-9b9edd9a9c1f admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:20.202248 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7425b-41ac-4dde-9b57-9b9edd9a9c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:20.202521 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7425b-41ac-4dde-9b57-9b9edd9a9c1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:20.203814 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7425b-41ac-4dde-9b57-9b9edd9a9c1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:51:20.204122 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7425b-41ac-4dde-9b57-9b9edd9a9c1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:20.205710 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7425b-41ac-4dde-9b57-9b9edd9a9c1f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:20.216970 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1415/2832] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:51:20.221990 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2784686-56ff-4b7e-8bf7-928b02826a9e None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:20.231146 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2784686-56ff-4b7e-8bf7-928b02826a9e None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:20.242576 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6aee9-3d9e-466f-ad51-3bde72054b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:20.242576 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6aee9-3d9e-466f-ad51-3bde72054b29 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:20.242576 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6aee9-3d9e-466f-ad51-3bde72054b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:20.242576 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6aee9-3d9e-466f-ad51-3bde72054b29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:20.242967 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6aee9-3d9e-466f-ad51-3bde72054b29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:20.242967 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6aee9-3d9e-466f-ad51-3bde72054b29 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:20.333659 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:20.340055 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1418/2833] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:51:20 2022] POST /identity/v3/projects => generated 393 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:51:20.345762 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7425b-41ac-4dde-9b57-9b9edd9a9c1f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:20.354038 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7425b-41ac-4dde-9b57-9b9edd9a9c1f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:20.415533 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b4fc9-6ce5-4fef-a7f4-012a01a789ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:20.416199 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b4fc9-6ce5-4fef-a7f4-012a01a789ef admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:20.416483 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b4fc9-6ce5-4fef-a7f4-012a01a789ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:20.416804 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b4fc9-6ce5-4fef-a7f4-012a01a789ef admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:20.417498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b4fc9-6ce5-4fef-a7f4-012a01a789ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:20.419440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b4fc9-6ce5-4fef-a7f4-012a01a789ef admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:20.494927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1416/2834] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:51:20 2022] POST /identity/v3/users => generated 297 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:51:20.499011 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6aee9-3d9e-466f-ad51-3bde72054b29 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:20.510072 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6aee9-3d9e-466f-ad51-3bde72054b29 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:20.514738 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd6230-cc83-47aa-92c6-045ac9be7073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:20.516050 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd6230-cc83-47aa-92c6-045ac9be7073 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:20.516364 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd6230-cc83-47aa-92c6-045ac9be7073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:20.516640 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd6230-cc83-47aa-92c6-045ac9be7073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:20.517359 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd6230-cc83-47aa-92c6-045ac9be7073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:20.518961 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd6230-cc83-47aa-92c6-045ac9be7073 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:20.529521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1419/2835] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:20.537734 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b4fc9-6ce5-4fef-a7f4-012a01a789ef admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:20.542968 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:20.548671 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b4fc9-6ce5-4fef-a7f4-012a01a789ef admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:20.553397 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603293c-738d-484f-853d-51db20532d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:20.554126 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603293c-738d-484f-853d-51db20532d2a admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:20.558045 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603293c-738d-484f-853d-51db20532d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:20.558045 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603293c-738d-484f-853d-51db20532d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:20.558045 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603293c-738d-484f-853d-51db20532d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:20.558045 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603293c-738d-484f-853d-51db20532d2a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:20.568079 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1417/2836] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:20.572111 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd6230-cc83-47aa-92c6-045ac9be7073 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:20.582862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd6230-cc83-47aa-92c6-045ac9be7073 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:20.591282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92e3d0-cac5-42ea-a430-48994948ef0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:20.591282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92e3d0-cac5-42ea-a430-48994948ef0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:20.591282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92e3d0-cac5-42ea-a430-48994948ef0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:20.591282 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92e3d0-cac5-42ea-a430-48994948ef0c admin admin] PATH_INFO: `/v3/projects/4c9a48a6458348ed97afeefe658512f8/users/744b36cc721d44938fa8234de0ed1524/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:20.603575 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92e3d0-cac5-42ea-a430-48994948ef0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c9a48a6458348ed97afeefe658512f8, user_id=744b36cc721d44938fa8234de0ed1524, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:20.606190 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92e3d0-cac5-42ea-a430-48994948ef0c admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:20.666546 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1420/2837] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:51:20 2022] PUT /identity/v3/projects/4c9a48a6458348ed97afeefe658512f8/users/744b36cc721d44938fa8234de0ed1524/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:20.677446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f31ec-8474-483c-82e2-09729fefbbaa None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:20.677752 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f31ec-8474-483c-82e2-09729fefbbaa None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:20.678032 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f31ec-8474-483c-82e2-09729fefbbaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:20.704815 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ec8f31ec-8474-483c-82e2-09729fefbbaa None None] MFA Rules not processed for user `744b36cc721d44938fa8234de0ed1524`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:20.706704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ec8f31ec-8474-483c-82e2-09729fefbbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:20.768859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1418/2838] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:20.773453 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92e3d0-cac5-42ea-a430-48994948ef0c admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:20.833901 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92e3d0-cac5-42ea-a430-48994948ef0c admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:20.838476 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238602fb-9432-4efa-815f-39f5e61a208b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:20.839151 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238602fb-9432-4efa-815f-39f5e61a208b admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:20.839435 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238602fb-9432-4efa-815f-39f5e61a208b admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:20.839711 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238602fb-9432-4efa-815f-39f5e61a208b admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:20.840393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238602fb-9432-4efa-815f-39f5e61a208b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:20.842343 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238602fb-9432-4efa-815f-39f5e61a208b admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:20.914372 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1421/2839] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:51:20 2022] POST /identity/v3/projects => generated 393 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:51:20.918383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f31ec-8474-483c-82e2-09729fefbbaa None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:20.930025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f31ec-8474-483c-82e2-09729fefbbaa None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:20.985921 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f8671-2c35-447e-8d1a-2acec18beeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:20.986590 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f8671-2c35-447e-8d1a-2acec18beeb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:20.986876 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f8671-2c35-447e-8d1a-2acec18beeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:20.987179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f8671-2c35-447e-8d1a-2acec18beeb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:20.987864 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f8671-2c35-447e-8d1a-2acec18beeb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:20.989769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f8671-2c35-447e-8d1a-2acec18beeb3 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:21.072084 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1419/2840] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:51:20 2022] POST /identity/v3/users => generated 303 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:51:21.076075 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238602fb-9432-4efa-815f-39f5e61a208b admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:21.086746 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238602fb-9432-4efa-815f-39f5e61a208b admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:21.091390 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e0a8e-3b62-4ce8-ab4d-76b73670d767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:21.092051 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e0a8e-3b62-4ce8-ab4d-76b73670d767 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:21.092333 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e0a8e-3b62-4ce8-ab4d-76b73670d767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:21.092655 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e0a8e-3b62-4ce8-ab4d-76b73670d767 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:21.093330 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e0a8e-3b62-4ce8-ab4d-76b73670d767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:21.095506 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e0a8e-3b62-4ce8-ab4d-76b73670d767 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:21.106027 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1422/2841] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:21.110769 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f8671-2c35-447e-8d1a-2acec18beeb3 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:21.120624 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f8671-2c35-447e-8d1a-2acec18beeb3 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:21.125149 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20761a-b65d-4275-9af8-47393227bfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:21.126015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20761a-b65d-4275-9af8-47393227bfde admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:21.126295 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20761a-b65d-4275-9af8-47393227bfde admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:21.126566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20761a-b65d-4275-9af8-47393227bfde admin admin] PATH_INFO: `/v3/projects/e810d764d3f74379b71d3b5005e14673/users/3827d44ff8524009a2458121dcf5a092/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:21.140954 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20761a-b65d-4275-9af8-47393227bfde admin admin] RBAC: Authorizing `identity:create_grant(project_id=e810d764d3f74379b71d3b5005e14673, user_id=3827d44ff8524009a2458121dcf5a092, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:21.143614 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20761a-b65d-4275-9af8-47393227bfde admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:21.241596 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1420/2842] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:51:21 2022] PUT /identity/v3/projects/e810d764d3f74379b71d3b5005e14673/users/3827d44ff8524009a2458121dcf5a092/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:21.245653 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e0a8e-3b62-4ce8-ab4d-76b73670d767 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:21.256365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e0a8e-3b62-4ce8-ab4d-76b73670d767 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:21.313430 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0d69b-f206-4801-8786-308e9617233a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:21.314088 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0d69b-f206-4801-8786-308e9617233a admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:21.314365 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0d69b-f206-4801-8786-308e9617233a admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:21.314635 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0d69b-f206-4801-8786-308e9617233a admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:21.315388 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0d69b-f206-4801-8786-308e9617233a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:21.316969 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0d69b-f206-4801-8786-308e9617233a admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:21.326774 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1423/2843] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:21.331159 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20761a-b65d-4275-9af8-47393227bfde admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:21.341422 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20761a-b65d-4275-9af8-47393227bfde admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:21.346714 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c7b12-8be7-4d6d-85c6-352107aff778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:21.347574 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c7b12-8be7-4d6d-85c6-352107aff778 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:21.353149 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c7b12-8be7-4d6d-85c6-352107aff778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:21.353149 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c7b12-8be7-4d6d-85c6-352107aff778 admin admin] PATH_INFO: `/v3/projects/e810d764d3f74379b71d3b5005e14673/users/3827d44ff8524009a2458121dcf5a092/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:21.353149 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c7b12-8be7-4d6d-85c6-352107aff778 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e810d764d3f74379b71d3b5005e14673, user_id=3827d44ff8524009a2458121dcf5a092, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:21.353149 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c7b12-8be7-4d6d-85c6-352107aff778 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:21.363280 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1421/2844] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:51:21 2022] PUT /identity/v3/projects/e810d764d3f74379b71d3b5005e14673/users/3827d44ff8524009a2458121dcf5a092/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:21.373214 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70927919-22fa-48be-a7f6-1643d2a83195 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:21.373505 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70927919-22fa-48be-a7f6-1643d2a83195 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:21.373781 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70927919-22fa-48be-a7f6-1643d2a83195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:21.400995 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-70927919-22fa-48be-a7f6-1643d2a83195 None None] MFA Rules not processed for user `3827d44ff8524009a2458121dcf5a092`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:21.402182 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-70927919-22fa-48be-a7f6-1643d2a83195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:21.431904 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:21.477454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1424/2845] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:21.488654 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c7b12-8be7-4d6d-85c6-352107aff778 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:21.490328 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ba4c7b12-8be7-4d6d-85c6-352107aff778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:21.526401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:21.548518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c7b12-8be7-4d6d-85c6-352107aff778 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:21.552288 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429242bf-cd85-4bd1-aec1-a25ec86583f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:21.552927 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429242bf-cd85-4bd1-aec1-a25ec86583f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:21.553205 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429242bf-cd85-4bd1-aec1-a25ec86583f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:21.553478 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429242bf-cd85-4bd1-aec1-a25ec86583f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:21.562965 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429242bf-cd85-4bd1-aec1-a25ec86583f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:21.564827 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429242bf-cd85-4bd1-aec1-a25ec86583f0 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:21.574190 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1422/2846] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:51:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:51:21.621740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70927919-22fa-48be-a7f6-1643d2a83195 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:21.623141 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-70927919-22fa-48be-a7f6-1643d2a83195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:21.636083 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70927919-22fa-48be-a7f6-1643d2a83195 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:21.640030 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88d1ac-31c6-4051-8288-9a2d033071a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:21.641367 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88d1ac-31c6-4051-8288-9a2d033071a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:21.641849 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88d1ac-31c6-4051-8288-9a2d033071a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:21.642128 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88d1ac-31c6-4051-8288-9a2d033071a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:21.698651 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88d1ac-31c6-4051-8288-9a2d033071a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:21.700119 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88d1ac-31c6-4051-8288-9a2d033071a3 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:21.709754 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1425/2847] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:51:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:51:26.456908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429242bf-cd85-4bd1-aec1-a25ec86583f0 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:26.521084 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429242bf-cd85-4bd1-aec1-a25ec86583f0 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:26.525783 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdd109-6169-42e1-a170-4d71227dcc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:26.526665 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdd109-6169-42e1-a170-4d71227dcc12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:26.526973 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdd109-6169-42e1-a170-4d71227dcc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:26.527248 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdd109-6169-42e1-a170-4d71227dcc12 admin admin] PATH_INFO: `/v3/users/744b36cc721d44938fa8234de0ed1524` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:26.528430 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdd109-6169-42e1-a170-4d71227dcc12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=744b36cc721d44938fa8234de0ed1524)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:26.530323 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdd109-6169-42e1-a170-4d71227dcc12 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:26.719096 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-efbdd109-6169-42e1-a170-4d71227dcc12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '744b36cc721d44938fa8234de0ed1524'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:26.783015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-efbdd109-6169-42e1-a170-4d71227dcc12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '744b36cc721d44938fa8234de0ed1524'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:26.783662 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-efbdd109-6169-42e1-a170-4d71227dcc12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '744b36cc721d44938fa8234de0ed1524'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:26.784269 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-efbdd109-6169-42e1-a170-4d71227dcc12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '744b36cc721d44938fa8234de0ed1524'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:26.792484 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-efbdd109-6169-42e1-a170-4d71227dcc12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '744b36cc721d44938fa8234de0ed1524'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:26.810165 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1423/2848] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:51:26 2022] DELETE /identity/v3/users/744b36cc721d44938fa8234de0ed1524 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:51:26.815463 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c8176-57cd-4962-b0e1-d1752f21f1db None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:26.815983 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c8176-57cd-4962-b0e1-d1752f21f1db None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:26.816088 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c8176-57cd-4962-b0e1-d1752f21f1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:26.832817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-889c8176-57cd-4962-b0e1-d1752f21f1db None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:26.834021 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-889c8176-57cd-4962-b0e1-d1752f21f1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:26.893171 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1426/2849] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:26.903680 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdd109-6169-42e1-a170-4d71227dcc12 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:26.904831 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-efbdd109-6169-42e1-a170-4d71227dcc12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:26.963585 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdd109-6169-42e1-a170-4d71227dcc12 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:26.967180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b43721-a2e9-44ee-95d1-6e83a8577f60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:26.967817 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b43721-a2e9-44ee-95d1-6e83a8577f60 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:26.968094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b43721-a2e9-44ee-95d1-6e83a8577f60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:26.968389 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b43721-a2e9-44ee-95d1-6e83a8577f60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:26.977426 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b43721-a2e9-44ee-95d1-6e83a8577f60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:26.978781 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b43721-a2e9-44ee-95d1-6e83a8577f60 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:26.982641 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:26.989343 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1424/2850] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:51:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:51:27.388090 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c8176-57cd-4962-b0e1-d1752f21f1db None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:27.395772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-889c8176-57cd-4962-b0e1-d1752f21f1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:27.403203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c8176-57cd-4962-b0e1-d1752f21f1db None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:27.409432 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad07c8-2a75-4ff6-9368-5c9ccf584f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:27.410294 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad07c8-2a75-4ff6-9368-5c9ccf584f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:27.410575 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad07c8-2a75-4ff6-9368-5c9ccf584f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:27.410850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad07c8-2a75-4ff6-9368-5c9ccf584f1c admin admin] PATH_INFO: `/v3/projects/4c9a48a6458348ed97afeefe658512f8` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:27.412071 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad07c8-2a75-4ff6-9368-5c9ccf584f1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c9a48a6458348ed97afeefe658512f8)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:27.414002 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad07c8-2a75-4ff6-9368-5c9ccf584f1c admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:27.512234 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7aad07c8-2a75-4ff6-9368-5c9ccf584f1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c9a48a6458348ed97afeefe658512f8', 'request_id': 'req-7aad07c8-2a75-4ff6-9368-5c9ccf584f1c'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:27.516627 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:27.518699 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7aad07c8-2a75-4ff6-9368-5c9ccf584f1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c9a48a6458348ed97afeefe658512f8', 'request_id': 'req-7aad07c8-2a75-4ff6-9368-5c9ccf584f1c'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:27.525189 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7aad07c8-2a75-4ff6-9368-5c9ccf584f1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c9a48a6458348ed97afeefe658512f8', 'request_id': 'req-7aad07c8-2a75-4ff6-9368-5c9ccf584f1c'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:27.625538 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7aad07c8-2a75-4ff6-9368-5c9ccf584f1c admin admin] The token cache is being invalidate because project 4c9a48a6458348ed97afeefe658512f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:51:27.625819 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7aad07c8-2a75-4ff6-9368-5c9ccf584f1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:27.626437 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-7aad07c8-2a75-4ff6-9368-5c9ccf584f1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:27.627988 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1427/2851] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:51:27 2022] DELETE /identity/v3/projects/4c9a48a6458348ed97afeefe658512f8 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:51:27.631924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b43721-a2e9-44ee-95d1-6e83a8577f60 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:27.643264 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b43721-a2e9-44ee-95d1-6e83a8577f60 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:27.701265 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327be60-8464-4167-8a3f-c4dccefba8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:27.702132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327be60-8464-4167-8a3f-c4dccefba8ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:27.702412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327be60-8464-4167-8a3f-c4dccefba8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:27.703370 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327be60-8464-4167-8a3f-c4dccefba8ba admin admin] PATH_INFO: `/v3/users/3827d44ff8524009a2458121dcf5a092` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:27.704565 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327be60-8464-4167-8a3f-c4dccefba8ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=3827d44ff8524009a2458121dcf5a092)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:27.706462 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327be60-8464-4167-8a3f-c4dccefba8ba admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:27.887799 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4327be60-8464-4167-8a3f-c4dccefba8ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3827d44ff8524009a2458121dcf5a092'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:27.974401 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4327be60-8464-4167-8a3f-c4dccefba8ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3827d44ff8524009a2458121dcf5a092'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:27.975027 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4327be60-8464-4167-8a3f-c4dccefba8ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3827d44ff8524009a2458121dcf5a092'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:27.975660 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4327be60-8464-4167-8a3f-c4dccefba8ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3827d44ff8524009a2458121dcf5a092'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:27.983008 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-4327be60-8464-4167-8a3f-c4dccefba8ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3827d44ff8524009a2458121dcf5a092'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:28.000838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1425/2852] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:51:27 2022] DELETE /identity/v3/users/3827d44ff8524009a2458121dcf5a092 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:28.382288 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad07c8-2a75-4ff6-9368-5c9ccf584f1c admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:28.383776 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-7aad07c8-2a75-4ff6-9368-5c9ccf584f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:28.400205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad07c8-2a75-4ff6-9368-5c9ccf584f1c admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:28.457487 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f44e2-3c9c-4b95-acae-6bce90e84a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:28.458340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f44e2-3c9c-4b95-acae-6bce90e84a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:28.458620 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f44e2-3c9c-4b95-acae-6bce90e84a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:28.458910 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f44e2-3c9c-4b95-acae-6bce90e84a53 admin admin] PATH_INFO: `/v3/projects/e810d764d3f74379b71d3b5005e14673` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:28.460099 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f44e2-3c9c-4b95-acae-6bce90e84a53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e810d764d3f74379b71d3b5005e14673)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:28.461981 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f44e2-3c9c-4b95-acae-6bce90e84a53 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:28.564562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a07f44e2-3c9c-4b95-acae-6bce90e84a53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e810d764d3f74379b71d3b5005e14673', 'request_id': 'req-a07f44e2-3c9c-4b95-acae-6bce90e84a53'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:28.569948 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a07f44e2-3c9c-4b95-acae-6bce90e84a53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e810d764d3f74379b71d3b5005e14673', 'request_id': 'req-a07f44e2-3c9c-4b95-acae-6bce90e84a53'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:28.576191 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a07f44e2-3c9c-4b95-acae-6bce90e84a53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e810d764d3f74379b71d3b5005e14673', 'request_id': 'req-a07f44e2-3c9c-4b95-acae-6bce90e84a53'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:28.643328 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a07f44e2-3c9c-4b95-acae-6bce90e84a53 admin admin] The token cache is being invalidate because project e810d764d3f74379b71d3b5005e14673 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:51:28.643525 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a07f44e2-3c9c-4b95-acae-6bce90e84a53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:28.644151 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a07f44e2-3c9c-4b95-acae-6bce90e84a53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:28.645678 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1428/2853] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:51:28 2022] DELETE /identity/v3/projects/e810d764d3f74379b71d3b5005e14673 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:28.651226 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13b225-250a-4c16-af63-b541d5ca3f53 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:28.651702 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13b225-250a-4c16-af63-b541d5ca3f53 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:28.651794 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13b225-250a-4c16-af63-b541d5ca3f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:28.669618 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-5b13b225-250a-4c16-af63-b541d5ca3f53 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:28.670802 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-5b13b225-250a-4c16-af63-b541d5ca3f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:28.729453 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1426/2854] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:28.737117 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee413ef4-f6f2-48c6-9a4c-b798bf28dd90 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:28.737406 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee413ef4-f6f2-48c6-9a4c-b798bf28dd90 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:28.737687 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee413ef4-f6f2-48c6-9a4c-b798bf28dd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:28.756238 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-ee413ef4-f6f2-48c6-9a4c-b798bf28dd90 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:28.757422 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ee413ef4-f6f2-48c6-9a4c-b798bf28dd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:28.765841 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1429/2855] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:28.769942 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13b225-250a-4c16-af63-b541d5ca3f53 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:28.779838 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13b225-250a-4c16-af63-b541d5ca3f53 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:28.784661 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99c3cec-9c04-4d1f-9a27-d0f898a40e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:28.785311 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c3cec-9c04-4d1f-9a27-d0f898a40e91 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:28.785597 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c3cec-9c04-4d1f-9a27-d0f898a40e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:28.785877 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c3cec-9c04-4d1f-9a27-d0f898a40e91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:28.786605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99c3cec-9c04-4d1f-9a27-d0f898a40e91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:51:28.786944 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99c3cec-9c04-4d1f-9a27-d0f898a40e91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:28.788536 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99c3cec-9c04-4d1f-9a27-d0f898a40e91 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:28.800325 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1427/2856] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:51:28.804528 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee413ef4-f6f2-48c6-9a4c-b798bf28dd90 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:28.814435 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee413ef4-f6f2-48c6-9a4c-b798bf28dd90 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:28.819095 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df3c25-a37b-46f6-8b7d-00a728b2304f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:28.819735 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df3c25-a37b-46f6-8b7d-00a728b2304f admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:28.820014 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df3c25-a37b-46f6-8b7d-00a728b2304f admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:28.820288 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df3c25-a37b-46f6-8b7d-00a728b2304f admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:28.820986 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df3c25-a37b-46f6-8b7d-00a728b2304f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:28.822940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df3c25-a37b-46f6-8b7d-00a728b2304f admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:28.910816 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1430/2857] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:51:28 2022] POST /identity/v3/projects => generated 401 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:51:28.914881 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99c3cec-9c04-4d1f-9a27-d0f898a40e91 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:28.921272 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:28.925181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99c3cec-9c04-4d1f-9a27-d0f898a40e91 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:28.981738 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c0d4b-9f8f-43f5-83f3-c2de44fa4c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:28.982451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c0d4b-9f8f-43f5-83f3-c2de44fa4c6a admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:28.983000 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c0d4b-9f8f-43f5-83f3-c2de44fa4c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:28.983304 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c0d4b-9f8f-43f5-83f3-c2de44fa4c6a admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:28.984763 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c0d4b-9f8f-43f5-83f3-c2de44fa4c6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:28.987183 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c0d4b-9f8f-43f5-83f3-c2de44fa4c6a admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:29.083420 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1428/2858] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:51:28 2022] POST /identity/v3/users => generated 301 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:51:29.087451 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df3c25-a37b-46f6-8b7d-00a728b2304f admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:29.097955 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df3c25-a37b-46f6-8b7d-00a728b2304f admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:29.102701 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa8d00-76ba-4074-bbe9-8dc756fa7c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:29.103431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa8d00-76ba-4074-bbe9-8dc756fa7c2e admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:29.103717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa8d00-76ba-4074-bbe9-8dc756fa7c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:29.103994 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa8d00-76ba-4074-bbe9-8dc756fa7c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:29.104722 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa8d00-76ba-4074-bbe9-8dc756fa7c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:29.106317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa8d00-76ba-4074-bbe9-8dc756fa7c2e admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:29.117280 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1431/2859] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:29.121001 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c0d4b-9f8f-43f5-83f3-c2de44fa4c6a admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:29.131627 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c0d4b-9f8f-43f5-83f3-c2de44fa4c6a admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:29.136222 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80204323-1a6a-403a-9099-5b4d2b1e00c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:29.136871 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80204323-1a6a-403a-9099-5b4d2b1e00c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:29.137151 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80204323-1a6a-403a-9099-5b4d2b1e00c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:29.137426 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80204323-1a6a-403a-9099-5b4d2b1e00c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:29.138138 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80204323-1a6a-403a-9099-5b4d2b1e00c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:29.139746 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80204323-1a6a-403a-9099-5b4d2b1e00c0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:29.149674 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1429/2860] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:29.153940 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa8d00-76ba-4074-bbe9-8dc756fa7c2e admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:29.163859 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa8d00-76ba-4074-bbe9-8dc756fa7c2e admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:29.168508 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993bc8b-9d17-47e7-b9e5-aee6711f73d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:29.169369 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993bc8b-9d17-47e7-b9e5-aee6711f73d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:29.169647 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993bc8b-9d17-47e7-b9e5-aee6711f73d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:29.169919 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993bc8b-9d17-47e7-b9e5-aee6711f73d6 admin admin] PATH_INFO: `/v3/projects/23a0b8d7af6b49adacf5dad536421ee0/users/e00541e25dd14d6b8776d912997e5035/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:29.184580 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993bc8b-9d17-47e7-b9e5-aee6711f73d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23a0b8d7af6b49adacf5dad536421ee0, user_id=e00541e25dd14d6b8776d912997e5035, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:29.187856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993bc8b-9d17-47e7-b9e5-aee6711f73d6 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:29.286362 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1432/2861] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:51:29 2022] PUT /identity/v3/projects/23a0b8d7af6b49adacf5dad536421ee0/users/e00541e25dd14d6b8776d912997e5035/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:51:29.297181 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df7309-cce0-4a19-8598-30147bafda06 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:29.297474 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df7309-cce0-4a19-8598-30147bafda06 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:29.297752 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df7309-cce0-4a19-8598-30147bafda06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:29.324041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-85df7309-cce0-4a19-8598-30147bafda06 None None] MFA Rules not processed for user `e00541e25dd14d6b8776d912997e5035`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:29.325246 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-85df7309-cce0-4a19-8598-30147bafda06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:29.388148 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1430/2862] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:29.392701 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993bc8b-9d17-47e7-b9e5-aee6711f73d6 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:29.402977 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:29.453705 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993bc8b-9d17-47e7-b9e5-aee6711f73d6 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:29.458498 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b5d20-d927-4d34-bc07-220d7e6eba4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:29.459202 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b5d20-d927-4d34-bc07-220d7e6eba4b admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:29.459487 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b5d20-d927-4d34-bc07-220d7e6eba4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:29.459765 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b5d20-d927-4d34-bc07-220d7e6eba4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:29.460465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b5d20-d927-4d34-bc07-220d7e6eba4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:29.462466 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b5d20-d927-4d34-bc07-220d7e6eba4b admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:29.551362 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1433/2863] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:51:29 2022] POST /identity/v3/projects => generated 399 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:51:29.555454 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df7309-cce0-4a19-8598-30147bafda06 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:29.566630 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df7309-cce0-4a19-8598-30147bafda06 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:29.623648 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1b52d-58c9-4011-bb2d-4016a7136e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:29.624349 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1b52d-58c9-4011-bb2d-4016a7136e3e admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:29.624734 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1b52d-58c9-4011-bb2d-4016a7136e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:29.625043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1b52d-58c9-4011-bb2d-4016a7136e3e admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:29.625748 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb1b52d-58c9-4011-bb2d-4016a7136e3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:29.627756 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb1b52d-58c9-4011-bb2d-4016a7136e3e admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:29.699974 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1431/2864] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:51:29 2022] POST /identity/v3/users => generated 306 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:51:29.704830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b5d20-d927-4d34-bc07-220d7e6eba4b admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:29.715244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b5d20-d927-4d34-bc07-220d7e6eba4b admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:29.720562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4edbc-8f83-444e-ab22-e38fdbff78f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:29.721219 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4edbc-8f83-444e-ab22-e38fdbff78f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:29.721499 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4edbc-8f83-444e-ab22-e38fdbff78f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:29.721773 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4edbc-8f83-444e-ab22-e38fdbff78f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:29.722494 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4edbc-8f83-444e-ab22-e38fdbff78f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:29.724125 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4edbc-8f83-444e-ab22-e38fdbff78f6 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:29.734914 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1434/2865] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:29.739094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1b52d-58c9-4011-bb2d-4016a7136e3e admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:29.744383 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:29.749791 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1b52d-58c9-4011-bb2d-4016a7136e3e admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:29.754517 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9bdf9-9efe-45dc-b6be-30d43c253db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:29.755469 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9bdf9-9efe-45dc-b6be-30d43c253db0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:29.755755 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9bdf9-9efe-45dc-b6be-30d43c253db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:29.756020 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9bdf9-9efe-45dc-b6be-30d43c253db0 admin admin] PATH_INFO: `/v3/projects/adec8d1459ff48bba491353e81381fd7/users/b4aa9f63f5f84eaaae89f65d5ca3676f/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:29.771147 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9bdf9-9efe-45dc-b6be-30d43c253db0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adec8d1459ff48bba491353e81381fd7, user_id=b4aa9f63f5f84eaaae89f65d5ca3676f, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:29.773887 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9bdf9-9efe-45dc-b6be-30d43c253db0 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:29.896231 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1432/2866] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:51:29 2022] PUT /identity/v3/projects/adec8d1459ff48bba491353e81381fd7/users/b4aa9f63f5f84eaaae89f65d5ca3676f/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:29.900436 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4edbc-8f83-444e-ab22-e38fdbff78f6 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:29.911017 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4edbc-8f83-444e-ab22-e38fdbff78f6 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:29.967209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd1d2b-f653-45f6-9534-b06b2f6abc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:29.967839 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd1d2b-f653-45f6-9534-b06b2f6abc70 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:29.968125 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd1d2b-f653-45f6-9534-b06b2f6abc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:29.968405 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd1d2b-f653-45f6-9534-b06b2f6abc70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:29.969160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfd1d2b-f653-45f6-9534-b06b2f6abc70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:29.971611 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfd1d2b-f653-45f6-9534-b06b2f6abc70 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:29.981587 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1435/2867] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:29.985778 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9bdf9-9efe-45dc-b6be-30d43c253db0 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:29.996175 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9bdf9-9efe-45dc-b6be-30d43c253db0 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:30.003337 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d4032-d704-4735-8dd1-49e77f63d3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:30.003337 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d4032-d704-4735-8dd1-49e77f63d3af admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:30.003337 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d4032-d704-4735-8dd1-49e77f63d3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:30.003337 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d4032-d704-4735-8dd1-49e77f63d3af admin admin] PATH_INFO: `/v3/projects/adec8d1459ff48bba491353e81381fd7/users/b4aa9f63f5f84eaaae89f65d5ca3676f/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:30.004553 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d4032-d704-4735-8dd1-49e77f63d3af admin admin] RBAC: Authorizing `identity:create_grant(project_id=adec8d1459ff48bba491353e81381fd7, user_id=b4aa9f63f5f84eaaae89f65d5ca3676f, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:30.007235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d4032-d704-4735-8dd1-49e77f63d3af admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:30.017988 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1433/2868] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:51:29 2022] PUT /identity/v3/projects/adec8d1459ff48bba491353e81381fd7/users/b4aa9f63f5f84eaaae89f65d5ca3676f/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:30.036227 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2733f-f890-4227-9030-e7b5f2a4426e None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:30.036516 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2733f-f890-4227-9030-e7b5f2a4426e None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:30.036793 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2733f-f890-4227-9030-e7b5f2a4426e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:30.064461 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-e7a2733f-f890-4227-9030-e7b5f2a4426e None None] MFA Rules not processed for user `b4aa9f63f5f84eaaae89f65d5ca3676f`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:30.065737 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-e7a2733f-f890-4227-9030-e7b5f2a4426e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:30.133054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1436/2869] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:30 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:30.144882 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d4032-d704-4735-8dd1-49e77f63d3af admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:30.146530 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-603d4032-d704-4735-8dd1-49e77f63d3af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:30.204461 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d4032-d704-4735-8dd1-49e77f63d3af admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:30.208238 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42192645-74b1-4e51-a383-4b60f8cabff0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:30.208877 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42192645-74b1-4e51-a383-4b60f8cabff0 service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:30.209161 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42192645-74b1-4e51-a383-4b60f8cabff0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:30.209435 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42192645-74b1-4e51-a383-4b60f8cabff0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:30.218571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42192645-74b1-4e51-a383-4b60f8cabff0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:30.220007 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42192645-74b1-4e51-a383-4b60f8cabff0 service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:30.229409 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1434/2870] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:51:30 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:51:30.483639 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2733f-f890-4227-9030-e7b5f2a4426e None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:30.490285 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:30.494080 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2733f-f890-4227-9030-e7b5f2a4426e None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:30.497690 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25672ef8-e69a-4c47-9b4e-0bdff20d53b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:30.501600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25672ef8-e69a-4c47-9b4e-0bdff20d53b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:30.501600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25672ef8-e69a-4c47-9b4e-0bdff20d53b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:30.501600 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25672ef8-e69a-4c47-9b4e-0bdff20d53b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:30.556777 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25672ef8-e69a-4c47-9b4e-0bdff20d53b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:30.558854 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25672ef8-e69a-4c47-9b4e-0bdff20d53b9 service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:30.568651 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1437/2871] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:51:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:51:35.430180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42192645-74b1-4e51-a383-4b60f8cabff0 service cinder] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:35.492651 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42192645-74b1-4e51-a383-4b60f8cabff0 service cinder] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:35.497211 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952dc03-0c61-490b-a5a7-55b75d128b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:35.498068 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952dc03-0c61-490b-a5a7-55b75d128b19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:35.498373 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952dc03-0c61-490b-a5a7-55b75d128b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:35.498644 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952dc03-0c61-490b-a5a7-55b75d128b19 admin admin] PATH_INFO: `/v3/users/e00541e25dd14d6b8776d912997e5035` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:35.499893 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952dc03-0c61-490b-a5a7-55b75d128b19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e00541e25dd14d6b8776d912997e5035)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:35.502283 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952dc03-0c61-490b-a5a7-55b75d128b19 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:35.685291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6952dc03-0c61-490b-a5a7-55b75d128b19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e00541e25dd14d6b8776d912997e5035'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:35.740053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6952dc03-0c61-490b-a5a7-55b75d128b19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e00541e25dd14d6b8776d912997e5035'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:35.740656 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6952dc03-0c61-490b-a5a7-55b75d128b19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e00541e25dd14d6b8776d912997e5035'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:35.741274 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6952dc03-0c61-490b-a5a7-55b75d128b19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e00541e25dd14d6b8776d912997e5035'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:35.748327 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6952dc03-0c61-490b-a5a7-55b75d128b19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e00541e25dd14d6b8776d912997e5035'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:35.753700 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:35.765605 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1435/2872] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:51:35 2022] DELETE /identity/v3/users/e00541e25dd14d6b8776d912997e5035 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:35.770697 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d18e-32ed-415d-8ed7-57a40e10aab6 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:35.771267 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d18e-32ed-415d-8ed7-57a40e10aab6 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:35.771384 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d18e-32ed-415d-8ed7-57a40e10aab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:35.789102 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-9792d18e-32ed-415d-8ed7-57a40e10aab6 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:35.790375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-9792d18e-32ed-415d-8ed7-57a40e10aab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:35.849428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1438/2873] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:35.859142 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952dc03-0c61-490b-a5a7-55b75d128b19 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:35.860710 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6952dc03-0c61-490b-a5a7-55b75d128b19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:35.917625 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952dc03-0c61-490b-a5a7-55b75d128b19 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:35.921304 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9901664-fa3e-40d3-a56e-5fbcf499ce23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:35.921945 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9901664-fa3e-40d3-a56e-5fbcf499ce23 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:35.922747 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9901664-fa3e-40d3-a56e-5fbcf499ce23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:35.923050 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9901664-fa3e-40d3-a56e-5fbcf499ce23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:35.931979 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9901664-fa3e-40d3-a56e-5fbcf499ce23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:35.933405 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9901664-fa3e-40d3-a56e-5fbcf499ce23 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:35.943332 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1436/2874] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:51:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:51:36.393260 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792d18e-32ed-415d-8ed7-57a40e10aab6 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:36.396044 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-9792d18e-32ed-415d-8ed7-57a40e10aab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:36.408009 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792d18e-32ed-415d-8ed7-57a40e10aab6 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:36.412667 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241ab95-817a-4ef9-aacc-262872dcef3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:36.413592 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241ab95-817a-4ef9-aacc-262872dcef3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:36.413994 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241ab95-817a-4ef9-aacc-262872dcef3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:36.414598 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241ab95-817a-4ef9-aacc-262872dcef3a admin admin] PATH_INFO: `/v3/projects/23a0b8d7af6b49adacf5dad536421ee0` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:36.415473 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241ab95-817a-4ef9-aacc-262872dcef3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=23a0b8d7af6b49adacf5dad536421ee0)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:36.419370 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241ab95-817a-4ef9-aacc-262872dcef3a admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:36.529209 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-1241ab95-817a-4ef9-aacc-262872dcef3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23a0b8d7af6b49adacf5dad536421ee0', 'request_id': 'req-1241ab95-817a-4ef9-aacc-262872dcef3a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:36.535672 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-1241ab95-817a-4ef9-aacc-262872dcef3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23a0b8d7af6b49adacf5dad536421ee0', 'request_id': 'req-1241ab95-817a-4ef9-aacc-262872dcef3a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:36.541851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-1241ab95-817a-4ef9-aacc-262872dcef3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23a0b8d7af6b49adacf5dad536421ee0', 'request_id': 'req-1241ab95-817a-4ef9-aacc-262872dcef3a'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:36.613693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-1241ab95-817a-4ef9-aacc-262872dcef3a admin admin] The token cache is being invalidate because project 23a0b8d7af6b49adacf5dad536421ee0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:51:36.613973 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-1241ab95-817a-4ef9-aacc-262872dcef3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:36.614585 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-1241ab95-817a-4ef9-aacc-262872dcef3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:36.616157 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1439/2875] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:51:36 2022] DELETE /identity/v3/projects/23a0b8d7af6b49adacf5dad536421ee0 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:36.620117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9901664-fa3e-40d3-a56e-5fbcf499ce23 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:36.629965 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9901664-fa3e-40d3-a56e-5fbcf499ce23 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:36.686097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9476f-5b4f-4540-abf1-d17c91cf719d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:36.686975 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9476f-5b4f-4540-abf1-d17c91cf719d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:36.687258 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9476f-5b4f-4540-abf1-d17c91cf719d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:36.687529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9476f-5b4f-4540-abf1-d17c91cf719d admin admin] PATH_INFO: `/v3/users/b4aa9f63f5f84eaaae89f65d5ca3676f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:36.688810 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd9476f-5b4f-4540-abf1-d17c91cf719d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4aa9f63f5f84eaaae89f65d5ca3676f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:36.690731 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd9476f-5b4f-4540-abf1-d17c91cf719d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:36.886105 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5bd9476f-5b4f-4540-abf1-d17c91cf719d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4aa9f63f5f84eaaae89f65d5ca3676f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:36.966204 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5bd9476f-5b4f-4540-abf1-d17c91cf719d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4aa9f63f5f84eaaae89f65d5ca3676f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:36.966841 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5bd9476f-5b4f-4540-abf1-d17c91cf719d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4aa9f63f5f84eaaae89f65d5ca3676f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:36.967456 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5bd9476f-5b4f-4540-abf1-d17c91cf719d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4aa9f63f5f84eaaae89f65d5ca3676f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:36.974926 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-5bd9476f-5b4f-4540-abf1-d17c91cf719d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4aa9f63f5f84eaaae89f65d5ca3676f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:36.992423 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1437/2876] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:51:36 2022] DELETE /identity/v3/users/b4aa9f63f5f84eaaae89f65d5ca3676f => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:37.402353 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241ab95-817a-4ef9-aacc-262872dcef3a admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:37.403817 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-1241ab95-817a-4ef9-aacc-262872dcef3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:37.417524 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241ab95-817a-4ef9-aacc-262872dcef3a admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:37.443721 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:37.474205 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55338e95-0b2a-48b9-8e28-2852c14f5de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:37.475086 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55338e95-0b2a-48b9-8e28-2852c14f5de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:37.475372 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55338e95-0b2a-48b9-8e28-2852c14f5de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:37.475648 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55338e95-0b2a-48b9-8e28-2852c14f5de4 admin admin] PATH_INFO: `/v3/projects/adec8d1459ff48bba491353e81381fd7` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:37.476820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55338e95-0b2a-48b9-8e28-2852c14f5de4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adec8d1459ff48bba491353e81381fd7)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:37.478715 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55338e95-0b2a-48b9-8e28-2852c14f5de4 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:37.608750 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-55338e95-0b2a-48b9-8e28-2852c14f5de4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adec8d1459ff48bba491353e81381fd7', 'request_id': 'req-55338e95-0b2a-48b9-8e28-2852c14f5de4'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:37.614912 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-55338e95-0b2a-48b9-8e28-2852c14f5de4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adec8d1459ff48bba491353e81381fd7', 'request_id': 'req-55338e95-0b2a-48b9-8e28-2852c14f5de4'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:37.621097 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-55338e95-0b2a-48b9-8e28-2852c14f5de4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adec8d1459ff48bba491353e81381fd7', 'request_id': 'req-55338e95-0b2a-48b9-8e28-2852c14f5de4'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:37.696358 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-55338e95-0b2a-48b9-8e28-2852c14f5de4 admin admin] The token cache is being invalidate because project adec8d1459ff48bba491353e81381fd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:51:37.696571 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-55338e95-0b2a-48b9-8e28-2852c14f5de4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:37.697200 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-55338e95-0b2a-48b9-8e28-2852c14f5de4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:37.698738 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1440/2877] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:51:37 2022] DELETE /identity/v3/projects/adec8d1459ff48bba491353e81381fd7 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:37.703669 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec899590-f4e5-42bb-8310-83d9f5885b30 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:37.704126 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec899590-f4e5-42bb-8310-83d9f5885b30 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:37.704284 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec899590-f4e5-42bb-8310-83d9f5885b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:37.724987 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-ec899590-f4e5-42bb-8310-83d9f5885b30 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:37.726196 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-ec899590-f4e5-42bb-8310-83d9f5885b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:37.730529 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:37.786827 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1438/2878] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:37.794675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f236c-cb84-4df5-9b3f-1210b62f1fda None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:37.795006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f236c-cb84-4df5-9b3f-1210b62f1fda None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:37.795330 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f236c-cb84-4df5-9b3f-1210b62f1fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:37.814026 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-5d5f236c-cb84-4df5-9b3f-1210b62f1fda None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:37.815295 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5d5f236c-cb84-4df5-9b3f-1210b62f1fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:37.823383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1441/2879] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:37.827670 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec899590-f4e5-42bb-8310-83d9f5885b30 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:37.837890 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec899590-f4e5-42bb-8310-83d9f5885b30 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:37.842396 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d16c7-39a0-496a-9149-d11ab74cd3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:37.843074 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d16c7-39a0-496a-9149-d11ab74cd3b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:37.843356 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d16c7-39a0-496a-9149-d11ab74cd3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:37.843627 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d16c7-39a0-496a-9149-d11ab74cd3b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:37.844839 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d16c7-39a0-496a-9149-d11ab74cd3b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:51:37.845140 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d16c7-39a0-496a-9149-d11ab74cd3b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:37.846712 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430d16c7-39a0-496a-9149-d11ab74cd3b3 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:37.857890 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1439/2880] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:51:37.862830 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f236c-cb84-4df5-9b3f-1210b62f1fda None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:37.872004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f236c-cb84-4df5-9b3f-1210b62f1fda None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:37.876669 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48fafb-0d3d-4d50-b381-d2f721d1d836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:37.877317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48fafb-0d3d-4d50-b381-d2f721d1d836 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:37.877601 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48fafb-0d3d-4d50-b381-d2f721d1d836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:37.877877 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c48fafb-0d3d-4d50-b381-d2f721d1d836 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:37.878575 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48fafb-0d3d-4d50-b381-d2f721d1d836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:37.880523 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c48fafb-0d3d-4d50-b381-d2f721d1d836 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:37.993136 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1442/2881] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:51:37 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:51:37.997147 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d16c7-39a0-496a-9149-d11ab74cd3b3 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:38.007090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d16c7-39a0-496a-9149-d11ab74cd3b3 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:38.063391 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b0a1a-e60b-4ea7-9585-3dfea62a2993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:38.064056 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b0a1a-e60b-4ea7-9585-3dfea62a2993 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:38.064335 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b0a1a-e60b-4ea7-9585-3dfea62a2993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:38.064606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b0a1a-e60b-4ea7-9585-3dfea62a2993 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:38.065281 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b0a1a-e60b-4ea7-9585-3dfea62a2993 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:38.067197 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1b0a1a-e60b-4ea7-9585-3dfea62a2993 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:38.143430 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1440/2882] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:51:37 2022] POST /identity/v3/users => generated 294 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:51:38.147509 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48fafb-0d3d-4d50-b381-d2f721d1d836 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:38.159089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c48fafb-0d3d-4d50-b381-d2f721d1d836 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:38.163688 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7f436-b768-422b-8090-50463f8de518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:38.164337 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7f436-b768-422b-8090-50463f8de518 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:38.164617 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7f436-b768-422b-8090-50463f8de518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:38.164888 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7f436-b768-422b-8090-50463f8de518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:38.165620 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7f436-b768-422b-8090-50463f8de518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:38.169288 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7f436-b768-422b-8090-50463f8de518 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:38.177734 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1443/2883] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:38.181554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b0a1a-e60b-4ea7-9585-3dfea62a2993 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:38.191600 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1b0a1a-e60b-4ea7-9585-3dfea62a2993 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:38.196263 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930d5fd-76d0-4662-b99a-696a44470be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:38.196926 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930d5fd-76d0-4662-b99a-696a44470be7 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:38.197207 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930d5fd-76d0-4662-b99a-696a44470be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:38.197486 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930d5fd-76d0-4662-b99a-696a44470be7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:38.198291 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930d5fd-76d0-4662-b99a-696a44470be7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:38.199836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9930d5fd-76d0-4662-b99a-696a44470be7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:38.210069 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1441/2884] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:38.213945 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7f436-b768-422b-8090-50463f8de518 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:38.224597 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7f436-b768-422b-8090-50463f8de518 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:38.229248 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5255d9-a585-4762-88c4-6303948d4ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:38.231466 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5255d9-a585-4762-88c4-6303948d4ba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:38.231466 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5255d9-a585-4762-88c4-6303948d4ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:38.231466 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5255d9-a585-4762-88c4-6303948d4ba4 admin admin] PATH_INFO: `/v3/projects/2731dd072cd64711821a89de370bb726/users/c3584ad46dcc4860b2ff745b8ad522d6/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:38.245262 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5255d9-a585-4762-88c4-6303948d4ba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2731dd072cd64711821a89de370bb726, user_id=c3584ad46dcc4860b2ff745b8ad522d6, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:38.247916 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5255d9-a585-4762-88c4-6303948d4ba4 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:38.366494 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1444/2885] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:51:38 2022] PUT /identity/v3/projects/2731dd072cd64711821a89de370bb726/users/c3584ad46dcc4860b2ff745b8ad522d6/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:51:38.377109 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f0f62-69db-4b85-a24f-382eb6de8913 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:38.377395 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f0f62-69db-4b85-a24f-382eb6de8913 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:38.377671 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f0f62-69db-4b85-a24f-382eb6de8913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:38.404423 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-387f0f62-69db-4b85-a24f-382eb6de8913 None None] MFA Rules not processed for user `c3584ad46dcc4860b2ff745b8ad522d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:38.405616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-387f0f62-69db-4b85-a24f-382eb6de8913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:38.468667 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1442/2886] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:38.473354 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5255d9-a585-4762-88c4-6303948d4ba4 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:38.525246 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:38.535484 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5255d9-a585-4762-88c4-6303948d4ba4 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:38.540101 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d2fb5-131c-4a48-99d9-20bad126e4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:38.540756 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d2fb5-131c-4a48-99d9-20bad126e4d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:38.541040 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d2fb5-131c-4a48-99d9-20bad126e4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:38.541324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d2fb5-131c-4a48-99d9-20bad126e4d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:38.542045 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d2fb5-131c-4a48-99d9-20bad126e4d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:38.543982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d2fb5-131c-4a48-99d9-20bad126e4d5 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:38.624372 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1445/2887] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:51:38 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:51:38.629180 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f0f62-69db-4b85-a24f-382eb6de8913 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:38.636025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:38.640076 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f0f62-69db-4b85-a24f-382eb6de8913 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:38.696697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02114187-09f4-462e-acbd-93320cc52717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:38.697347 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02114187-09f4-462e-acbd-93320cc52717 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:38.697626 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02114187-09f4-462e-acbd-93320cc52717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:38.697902 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02114187-09f4-462e-acbd-93320cc52717 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:38.698576 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02114187-09f4-462e-acbd-93320cc52717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:38.700512 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02114187-09f4-462e-acbd-93320cc52717 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:38.794407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1443/2888] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:51:38 2022] POST /identity/v3/users => generated 300 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:51:38.799244 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d2fb5-131c-4a48-99d9-20bad126e4d5 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:38.809440 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d2fb5-131c-4a48-99d9-20bad126e4d5 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:38.814943 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f417eb6-9265-4f12-8bd9-1f6996c677c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:38.816048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f417eb6-9265-4f12-8bd9-1f6996c677c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:38.816048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f417eb6-9265-4f12-8bd9-1f6996c677c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:38.816379 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f417eb6-9265-4f12-8bd9-1f6996c677c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:38.816829 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f417eb6-9265-4f12-8bd9-1f6996c677c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:38.818419 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f417eb6-9265-4f12-8bd9-1f6996c677c8 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:38.828992 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1446/2889] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:38.833229 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02114187-09f4-462e-acbd-93320cc52717 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:38.843136 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02114187-09f4-462e-acbd-93320cc52717 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:38.848250 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51d4a3-4644-4b0a-bd34-2308b49c74e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:38.849109 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51d4a3-4644-4b0a-bd34-2308b49c74e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:38.849384 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51d4a3-4644-4b0a-bd34-2308b49c74e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:38.849654 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51d4a3-4644-4b0a-bd34-2308b49c74e7 admin admin] PATH_INFO: `/v3/projects/298f66a9790d455d93b31cb829744efa/users/22fc774a5d6f4dbeb380a725dae0fa2f/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:38.864924 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51d4a3-4644-4b0a-bd34-2308b49c74e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=298f66a9790d455d93b31cb829744efa, user_id=22fc774a5d6f4dbeb380a725dae0fa2f, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:38.867583 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51d4a3-4644-4b0a-bd34-2308b49c74e7 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:38.944374 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1444/2890] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:51:38 2022] PUT /identity/v3/projects/298f66a9790d455d93b31cb829744efa/users/22fc774a5d6f4dbeb380a725dae0fa2f/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:38.948311 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f417eb6-9265-4f12-8bd9-1f6996c677c8 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:38.958391 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f417eb6-9265-4f12-8bd9-1f6996c677c8 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:39.018628 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509e01f-c537-44cf-a39b-ae1724b3cf3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:39.019497 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509e01f-c537-44cf-a39b-ae1724b3cf3f admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:39.019852 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509e01f-c537-44cf-a39b-ae1724b3cf3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:39.020136 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509e01f-c537-44cf-a39b-ae1724b3cf3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:39.020856 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509e01f-c537-44cf-a39b-ae1724b3cf3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:39.022530 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509e01f-c537-44cf-a39b-ae1724b3cf3f admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:39.037859 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1447/2891] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:39.038548 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51d4a3-4644-4b0a-bd34-2308b49c74e7 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:39.050342 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51d4a3-4644-4b0a-bd34-2308b49c74e7 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:39.058106 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7f444-85bb-4502-87fd-75168229548d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:39.059046 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7f444-85bb-4502-87fd-75168229548d admin admin] REQUEST_METHOD: `PUT` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:39.059333 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7f444-85bb-4502-87fd-75168229548d admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:39.059612 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7f444-85bb-4502-87fd-75168229548d admin admin] PATH_INFO: `/v3/projects/298f66a9790d455d93b31cb829744efa/users/22fc774a5d6f4dbeb380a725dae0fa2f/roles/0efeff46573247f3b9f1e2dec2968672` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:39.061990 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7f444-85bb-4502-87fd-75168229548d admin admin] RBAC: Authorizing `identity:create_grant(project_id=298f66a9790d455d93b31cb829744efa, user_id=22fc774a5d6f4dbeb380a725dae0fa2f, role_id=0efeff46573247f3b9f1e2dec2968672)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:39.065867 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7f444-85bb-4502-87fd-75168229548d admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:39.076863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1445/2892] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:51:39 2022] PUT /identity/v3/projects/298f66a9790d455d93b31cb829744efa/users/22fc774a5d6f4dbeb380a725dae0fa2f/roles/0efeff46573247f3b9f1e2dec2968672 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:39.088588 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b950e89-be6d-4acb-a836-183452dd081d None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:39.088880 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b950e89-be6d-4acb-a836-183452dd081d None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:39.089158 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b950e89-be6d-4acb-a836-183452dd081d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:39.118958 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-6b950e89-be6d-4acb-a836-183452dd081d None None] MFA Rules not processed for user `22fc774a5d6f4dbeb380a725dae0fa2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:39.120203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-6b950e89-be6d-4acb-a836-183452dd081d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:39.192896 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1448/2893] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:39.203623 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7f444-85bb-4502-87fd-75168229548d admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:39.205482 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3ce7f444-85bb-4502-87fd-75168229548d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:39.267240 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7f444-85bb-4502-87fd-75168229548d admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:39.270859 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a883af-7f3e-41e4-bcd7-b02ac2a4cdeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:39.271540 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a883af-7f3e-41e4-bcd7-b02ac2a4cdeb service cinder] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:39.271818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a883af-7f3e-41e4-bcd7-b02ac2a4cdeb service cinder] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:39.272088 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a883af-7f3e-41e4-bcd7-b02ac2a4cdeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:39.282725 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a883af-7f3e-41e4-bcd7-b02ac2a4cdeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:39.284198 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a883af-7f3e-41e4-bcd7-b02ac2a4cdeb service cinder] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:39.294126 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1446/2894] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:51:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:51:40.541723 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b950e89-be6d-4acb-a836-183452dd081d None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:40.591772 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:40.606218 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b950e89-be6d-4acb-a836-183452dd081d None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:40.610797 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12996f-94c4-40d3-abf6-921cdb8de354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:40.612203 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12996f-94c4-40d3-abf6-921cdb8de354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:40.612512 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12996f-94c4-40d3-abf6-921cdb8de354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:40.612786 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12996f-94c4-40d3-abf6-921cdb8de354 admin admin] PATH_INFO: `/v3/users/c3584ad46dcc4860b2ff745b8ad522d6` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:40.613981 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12996f-94c4-40d3-abf6-921cdb8de354 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3584ad46dcc4860b2ff745b8ad522d6)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:40.615979 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12996f-94c4-40d3-abf6-921cdb8de354 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:40.810226 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5d12996f-94c4-40d3-abf6-921cdb8de354 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3584ad46dcc4860b2ff745b8ad522d6'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:40.866040 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5d12996f-94c4-40d3-abf6-921cdb8de354 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3584ad46dcc4860b2ff745b8ad522d6'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:40.874577 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5d12996f-94c4-40d3-abf6-921cdb8de354 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3584ad46dcc4860b2ff745b8ad522d6'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:40.896042 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5d12996f-94c4-40d3-abf6-921cdb8de354 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3584ad46dcc4860b2ff745b8ad522d6'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:40.901724 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-5d12996f-94c4-40d3-abf6-921cdb8de354 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3584ad46dcc4860b2ff745b8ad522d6'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:40.912787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1449/2895] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:51:40 2022] DELETE /identity/v3/users/c3584ad46dcc4860b2ff745b8ad522d6 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:40.914608 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7ad8e-38a5-49e9-815c-49dc6153cd20 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:40.915075 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7ad8e-38a5-49e9-815c-49dc6153cd20 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:40.915210 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7ad8e-38a5-49e9-815c-49dc6153cd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:40.937132 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-18b7ad8e-38a5-49e9-815c-49dc6153cd20 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:40.938293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-18b7ad8e-38a5-49e9-815c-49dc6153cd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:40.999210 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:41.008500 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1447/2896] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:41.018694 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12996f-94c4-40d3-abf6-921cdb8de354 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:41.020264 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-5d12996f-94c4-40d3-abf6-921cdb8de354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:41.082862 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12996f-94c4-40d3-abf6-921cdb8de354 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:41.086562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959716a0-b307-4ac9-b373-702e618fd921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:41.087297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959716a0-b307-4ac9-b373-702e618fd921 service neutron] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:41.087581 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959716a0-b307-4ac9-b373-702e618fd921 service neutron] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:41.087840 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959716a0-b307-4ac9-b373-702e618fd921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:41.097273 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959716a0-b307-4ac9-b373-702e618fd921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:41.098716 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959716a0-b307-4ac9-b373-702e618fd921 service neutron] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:41.112584 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1450/2897] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:51:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:51:41.143894 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7ad8e-38a5-49e9-815c-49dc6153cd20 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:41.145421 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-18b7ad8e-38a5-49e9-815c-49dc6153cd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:41.232652 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7ad8e-38a5-49e9-815c-49dc6153cd20 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:41.239185 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496522c-327b-4316-b3a2-2f88709a9028 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f2e241494244edf8d283e8f44192c46', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:41.239844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496522c-327b-4316-b3a2-2f88709a9028 service nova] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:41.240129 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496522c-327b-4316-b3a2-2f88709a9028 service nova] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:41.240404 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496522c-327b-4316-b3a2-2f88709a9028 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:41.244037 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-2496522c-327b-4316-b3a2-2f88709a9028 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:41.259565 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496522c-327b-4316-b3a2-2f88709a9028 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:41.261167 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496522c-327b-4316-b3a2-2f88709a9028 service nova] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:41.272960 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1448/2898] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:51:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:51:41.649053 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959716a0-b307-4ac9-b373-702e618fd921 service neutron] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:41.655479 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-959716a0-b307-4ac9-b373-702e618fd921 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:41.695462 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959716a0-b307-4ac9-b373-702e618fd921 service neutron] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:41.698445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fe00f-c14b-4c73-9a67-60c811e951c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:41.699561 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fe00f-c14b-4c73-9a67-60c811e951c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:41.699914 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fe00f-c14b-4c73-9a67-60c811e951c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:41.700258 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fe00f-c14b-4c73-9a67-60c811e951c1 admin admin] PATH_INFO: `/v3/projects/2731dd072cd64711821a89de370bb726` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:41.701671 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493fe00f-c14b-4c73-9a67-60c811e951c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2731dd072cd64711821a89de370bb726)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:41.703864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493fe00f-c14b-4c73-9a67-60c811e951c1 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:41.872291 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-493fe00f-c14b-4c73-9a67-60c811e951c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2731dd072cd64711821a89de370bb726', 'request_id': 'req-493fe00f-c14b-4c73-9a67-60c811e951c1'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:41.883383 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-493fe00f-c14b-4c73-9a67-60c811e951c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2731dd072cd64711821a89de370bb726', 'request_id': 'req-493fe00f-c14b-4c73-9a67-60c811e951c1'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:41.891318 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-493fe00f-c14b-4c73-9a67-60c811e951c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2731dd072cd64711821a89de370bb726', 'request_id': 'req-493fe00f-c14b-4c73-9a67-60c811e951c1'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:41.895437 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:42.003558 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-493fe00f-c14b-4c73-9a67-60c811e951c1 admin admin] The token cache is being invalidate because project 2731dd072cd64711821a89de370bb726 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:51:42.007336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-493fe00f-c14b-4c73-9a67-60c811e951c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:42.007336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-493fe00f-c14b-4c73-9a67-60c811e951c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:42.007336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1451/2899] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:51:41 2022] DELETE /identity/v3/projects/2731dd072cd64711821a89de370bb726 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:42.010145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496522c-327b-4316-b3a2-2f88709a9028 service nova] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:42.021178 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496522c-327b-4316-b3a2-2f88709a9028 service nova] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:42.083250 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc019085-84f6-4307-9397-347be32a2e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:42.084150 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc019085-84f6-4307-9397-347be32a2e09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:42.084433 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc019085-84f6-4307-9397-347be32a2e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:42.084708 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc019085-84f6-4307-9397-347be32a2e09 admin admin] PATH_INFO: `/v3/users/22fc774a5d6f4dbeb380a725dae0fa2f` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:42.086130 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc019085-84f6-4307-9397-347be32a2e09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22fc774a5d6f4dbeb380a725dae0fa2f)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:42.088112 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc019085-84f6-4307-9397-347be32a2e09 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:42.287191 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-bc019085-84f6-4307-9397-347be32a2e09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22fc774a5d6f4dbeb380a725dae0fa2f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:42.360588 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-bc019085-84f6-4307-9397-347be32a2e09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22fc774a5d6f4dbeb380a725dae0fa2f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:42.362602 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-bc019085-84f6-4307-9397-347be32a2e09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22fc774a5d6f4dbeb380a725dae0fa2f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:42.364642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-bc019085-84f6-4307-9397-347be32a2e09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22fc774a5d6f4dbeb380a725dae0fa2f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:42.372948 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-bc019085-84f6-4307-9397-347be32a2e09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22fc774a5d6f4dbeb380a725dae0fa2f'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:42.399724 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1449/2900] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:51:42 2022] DELETE /identity/v3/users/22fc774a5d6f4dbeb380a725dae0fa2f => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:42.857116 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fe00f-c14b-4c73-9a67-60c811e951c1 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:42.858661 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-493fe00f-c14b-4c73-9a67-60c811e951c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:42.879313 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fe00f-c14b-4c73-9a67-60c811e951c1 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:42.952092 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3a81b-8f25-4850-ad88-26c6d5467b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:42.952950 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3a81b-8f25-4850-ad88-26c6d5467b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:42.953344 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3a81b-8f25-4850-ad88-26c6d5467b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:42.953631 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3a81b-8f25-4850-ad88-26c6d5467b88 admin admin] PATH_INFO: `/v3/projects/298f66a9790d455d93b31cb829744efa` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:42.955602 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3a81b-8f25-4850-ad88-26c6d5467b88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=298f66a9790d455d93b31cb829744efa)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:42.957555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d3a81b-8f25-4850-ad88-26c6d5467b88 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:43.075603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-83d3a81b-8f25-4850-ad88-26c6d5467b88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '298f66a9790d455d93b31cb829744efa', 'request_id': 'req-83d3a81b-8f25-4850-ad88-26c6d5467b88'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:43.081469 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-83d3a81b-8f25-4850-ad88-26c6d5467b88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '298f66a9790d455d93b31cb829744efa', 'request_id': 'req-83d3a81b-8f25-4850-ad88-26c6d5467b88'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:43.088742 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-83d3a81b-8f25-4850-ad88-26c6d5467b88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '298f66a9790d455d93b31cb829744efa', 'request_id': 'req-83d3a81b-8f25-4850-ad88-26c6d5467b88'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:43.183173 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-83d3a81b-8f25-4850-ad88-26c6d5467b88 admin admin] The token cache is being invalidate because project 298f66a9790d455d93b31cb829744efa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:51:43.183463 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-83d3a81b-8f25-4850-ad88-26c6d5467b88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:43.184180 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-83d3a81b-8f25-4850-ad88-26c6d5467b88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:43.185713 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1452/2901] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:51:42 2022] DELETE /identity/v3/projects/298f66a9790d455d93b31cb829744efa => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:43.190718 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beac4ec-f201-471d-84b7-7cc541cc7af4 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:43.191227 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beac4ec-f201-471d-84b7-7cc541cc7af4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:43.191328 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beac4ec-f201-471d-84b7-7cc541cc7af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:43.199334 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:43.210064 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-0beac4ec-f201-471d-84b7-7cc541cc7af4 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:43.211322 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-0beac4ec-f201-471d-84b7-7cc541cc7af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:43.271407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1450/2902] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:43.279425 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51757e-e1bd-4d66-83ba-0af25a7c766c None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:43.279740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51757e-e1bd-4d66-83ba-0af25a7c766c None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:43.280021 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51757e-e1bd-4d66-83ba-0af25a7c766c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:43.299208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-1e51757e-e1bd-4d66-83ba-0af25a7c766c None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:43.300408 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-1e51757e-e1bd-4d66-83ba-0af25a7c766c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:43.308548 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1453/2903] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:43.313035 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beac4ec-f201-471d-84b7-7cc541cc7af4 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:43.324534 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beac4ec-f201-471d-84b7-7cc541cc7af4 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:43.329298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba752769-4c59-4534-b162-e4143867b4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:43.330697 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba752769-4c59-4534-b162-e4143867b4e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:43.331025 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba752769-4c59-4534-b162-e4143867b4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:43.331300 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba752769-4c59-4534-b162-e4143867b4e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:43.332029 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba752769-4c59-4534-b162-e4143867b4e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:51:43.332336 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba752769-4c59-4534-b162-e4143867b4e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:43.334061 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba752769-4c59-4534-b162-e4143867b4e2 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:43.346576 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1451/2904] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:51:43.351375 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51757e-e1bd-4d66-83ba-0af25a7c766c None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:43.361243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51757e-e1bd-4d66-83ba-0af25a7c766c None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:43.366091 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1687ea1-6396-4b8d-802a-11dd23da5a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:43.366735 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1687ea1-6396-4b8d-802a-11dd23da5a4b admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:43.367059 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1687ea1-6396-4b8d-802a-11dd23da5a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:43.367340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1687ea1-6396-4b8d-802a-11dd23da5a4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:43.368551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1687ea1-6396-4b8d-802a-11dd23da5a4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:43.370167 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1687ea1-6396-4b8d-802a-11dd23da5a4b admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:43.469929 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1454/2905] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:51:43 2022] POST /identity/v3/projects => generated 359 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:51:43.474804 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba752769-4c59-4534-b162-e4143867b4e2 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:43.484917 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba752769-4c59-4534-b162-e4143867b4e2 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:43.540348 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004ac04-b1c3-4bfe-a524-9a395949a043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:43.541016 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004ac04-b1c3-4bfe-a524-9a395949a043 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:43.541297 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004ac04-b1c3-4bfe-a524-9a395949a043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:43.541585 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004ac04-b1c3-4bfe-a524-9a395949a043 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:43.542272 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004ac04-b1c3-4bfe-a524-9a395949a043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:43.544206 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004ac04-b1c3-4bfe-a524-9a395949a043 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:43.617250 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1452/2906] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:51:43 2022] POST /identity/v3/users => generated 280 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:51:43.621442 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1687ea1-6396-4b8d-802a-11dd23da5a4b admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:43.632362 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1687ea1-6396-4b8d-802a-11dd23da5a4b admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:43.637294 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a406e5-4626-48dd-a312-7c31acab01f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:43.638008 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a406e5-4626-48dd-a312-7c31acab01f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:43.638297 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a406e5-4626-48dd-a312-7c31acab01f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:43.638574 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a406e5-4626-48dd-a312-7c31acab01f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:43.639340 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a406e5-4626-48dd-a312-7c31acab01f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:43.640932 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a406e5-4626-48dd-a312-7c31acab01f3 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:43.652472 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1455/2907] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:43.656449 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004ac04-b1c3-4bfe-a524-9a395949a043 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:43.666424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004ac04-b1c3-4bfe-a524-9a395949a043 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:43.671201 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06850d58-ec48-488e-85ec-8fb9fbea12df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:43.671851 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06850d58-ec48-488e-85ec-8fb9fbea12df admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:43.672398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06850d58-ec48-488e-85ec-8fb9fbea12df admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:43.672682 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06850d58-ec48-488e-85ec-8fb9fbea12df admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:43.673977 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06850d58-ec48-488e-85ec-8fb9fbea12df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:43.675866 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06850d58-ec48-488e-85ec-8fb9fbea12df admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:43.685783 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1453/2908] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:43.690521 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a406e5-4626-48dd-a312-7c31acab01f3 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:43.700448 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a406e5-4626-48dd-a312-7c31acab01f3 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:43.705641 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1522618-a65d-4c9c-9f9f-cc7527f5cf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:43.706501 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1522618-a65d-4c9c-9f9f-cc7527f5cf09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:43.706779 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1522618-a65d-4c9c-9f9f-cc7527f5cf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:43.707081 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1522618-a65d-4c9c-9f9f-cc7527f5cf09 admin admin] PATH_INFO: `/v3/projects/e6c36c4d39ff4d22a2c4d9a79558e10e/users/6a4781bdd7e448b780dfbec9b30d2b21/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:43.721436 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1522618-a65d-4c9c-9f9f-cc7527f5cf09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6c36c4d39ff4d22a2c4d9a79558e10e, user_id=6a4781bdd7e448b780dfbec9b30d2b21, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:43.724110 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1522618-a65d-4c9c-9f9f-cc7527f5cf09 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:43.794284 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1456/2909] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:51:43 2022] PUT /identity/v3/projects/e6c36c4d39ff4d22a2c4d9a79558e10e/users/6a4781bdd7e448b780dfbec9b30d2b21/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:43.806159 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0f2e3-570e-40d8-8a81-6f6f6ff52081 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:43.806446 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0f2e3-570e-40d8-8a81-6f6f6ff52081 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:43.806740 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0f2e3-570e-40d8-8a81-6f6f6ff52081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:43.834555 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-54b0f2e3-570e-40d8-8a81-6f6f6ff52081 None None] MFA Rules not processed for user `6a4781bdd7e448b780dfbec9b30d2b21`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:43.835809 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-54b0f2e3-570e-40d8-8a81-6f6f6ff52081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:43.904910 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:43.905150 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1454/2910] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:43 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:43.937745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1522618-a65d-4c9c-9f9f-cc7527f5cf09 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:43.940484 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b1522618-a65d-4c9c-9f9f-cc7527f5cf09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:43.962377 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:43.997134 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1522618-a65d-4c9c-9f9f-cc7527f5cf09 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:44.000727 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de326a9e-9e70-466d-a62d-f171c5190d4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:44.001459 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de326a9e-9e70-466d-a62d-f171c5190d4c service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:44.002454 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de326a9e-9e70-466d-a62d-f171c5190d4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:44.002864 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de326a9e-9e70-466d-a62d-f171c5190d4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:44.011322 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de326a9e-9e70-466d-a62d-f171c5190d4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:44.012740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de326a9e-9e70-466d-a62d-f171c5190d4c service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:44.022368 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1457/2911] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:51:43 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:51:44.040125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0f2e3-570e-40d8-8a81-6f6f6ff52081 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:44.107642 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0f2e3-570e-40d8-8a81-6f6f6ff52081 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:44.113400 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f7295-e120-49ef-9a1b-8f3375a28f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:44.114293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f7295-e120-49ef-9a1b-8f3375a28f32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:44.114644 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f7295-e120-49ef-9a1b-8f3375a28f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:44.114917 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f7295-e120-49ef-9a1b-8f3375a28f32 admin admin] PATH_INFO: `/v3/users/6a4781bdd7e448b780dfbec9b30d2b21` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:44.116855 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f7295-e120-49ef-9a1b-8f3375a28f32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a4781bdd7e448b780dfbec9b30d2b21)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:44.118871 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f7295-e120-49ef-9a1b-8f3375a28f32 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:44.307278 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fe9f7295-e120-49ef-9a1b-8f3375a28f32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a4781bdd7e448b780dfbec9b30d2b21'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:44.403081 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fe9f7295-e120-49ef-9a1b-8f3375a28f32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a4781bdd7e448b780dfbec9b30d2b21'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:44.403704 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fe9f7295-e120-49ef-9a1b-8f3375a28f32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a4781bdd7e448b780dfbec9b30d2b21'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:44.404298 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fe9f7295-e120-49ef-9a1b-8f3375a28f32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a4781bdd7e448b780dfbec9b30d2b21'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:44.412218 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-fe9f7295-e120-49ef-9a1b-8f3375a28f32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a4781bdd7e448b780dfbec9b30d2b21'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:44.429067 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1455/2912] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:51:44 2022] DELETE /identity/v3/users/6a4781bdd7e448b780dfbec9b30d2b21 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:51:44.433611 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b4497-604c-4ef1-af33-da963039b1e7 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:44.433902 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b4497-604c-4ef1-af33-da963039b1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:44.434206 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b4497-604c-4ef1-af33-da963039b1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:44.451517 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-826b4497-604c-4ef1-af33-da963039b1e7 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:44.452659 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-826b4497-604c-4ef1-af33-da963039b1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:44.510603 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1458/2913] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:44.521612 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f7295-e120-49ef-9a1b-8f3375a28f32 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:44.523052 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-fe9f7295-e120-49ef-9a1b-8f3375a28f32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:44.580312 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f7295-e120-49ef-9a1b-8f3375a28f32 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:44.586248 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536404a-19e6-4203-b1e9-672d84fceb64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:44.586905 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536404a-19e6-4203-b1e9-672d84fceb64 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:44.587206 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536404a-19e6-4203-b1e9-672d84fceb64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:44.590533 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536404a-19e6-4203-b1e9-672d84fceb64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:44.597051 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536404a-19e6-4203-b1e9-672d84fceb64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:44.598471 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536404a-19e6-4203-b1e9-672d84fceb64 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:44.607982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1456/2914] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:51:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:51:45.007797 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b4497-604c-4ef1-af33-da963039b1e7 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:45.009267 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-826b4497-604c-4ef1-af33-da963039b1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:45.023841 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b4497-604c-4ef1-af33-da963039b1e7 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:45.029693 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9526954a-8588-489c-8a9a-e25396e13c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:45.030745 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526954a-8588-489c-8a9a-e25396e13c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:45.031058 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526954a-8588-489c-8a9a-e25396e13c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:45.031334 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526954a-8588-489c-8a9a-e25396e13c6b admin admin] PATH_INFO: `/v3/projects/e6c36c4d39ff4d22a2c4d9a79558e10e` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:45.032642 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9526954a-8588-489c-8a9a-e25396e13c6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6c36c4d39ff4d22a2c4d9a79558e10e)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:45.035853 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9526954a-8588-489c-8a9a-e25396e13c6b admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:45.126339 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-9526954a-8588-489c-8a9a-e25396e13c6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6c36c4d39ff4d22a2c4d9a79558e10e', 'request_id': 'req-9526954a-8588-489c-8a9a-e25396e13c6b'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:45.131977 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-9526954a-8588-489c-8a9a-e25396e13c6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6c36c4d39ff4d22a2c4d9a79558e10e', 'request_id': 'req-9526954a-8588-489c-8a9a-e25396e13c6b'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:45.138048 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-9526954a-8588-489c-8a9a-e25396e13c6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6c36c4d39ff4d22a2c4d9a79558e10e', 'request_id': 'req-9526954a-8588-489c-8a9a-e25396e13c6b'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:45.207763 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-9526954a-8588-489c-8a9a-e25396e13c6b admin admin] The token cache is being invalidate because project e6c36c4d39ff4d22a2c4d9a79558e10e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:51:45.208208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-9526954a-8588-489c-8a9a-e25396e13c6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:45.208840 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-9526954a-8588-489c-8a9a-e25396e13c6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:45.210380 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1459/2915] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:51:45 2022] DELETE /identity/v3/projects/e6c36c4d39ff4d22a2c4d9a79558e10e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:45.215294 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7ecf3-f512-4bec-9180-939aee0e0335 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:45.215585 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7ecf3-f512-4bec-9180-939aee0e0335 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:45.215862 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7ecf3-f512-4bec-9180-939aee0e0335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:45.233794 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-22e7ecf3-f512-4bec-9180-939aee0e0335 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:45.241940 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-22e7ecf3-f512-4bec-9180-939aee0e0335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:45.253127 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:45.295616 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1457/2916] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:45.304089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9aef0c-12f5-48c7-b09c-7b14caf22ca6 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:45.304432 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9aef0c-12f5-48c7-b09c-7b14caf22ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:45.304725 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9aef0c-12f5-48c7-b09c-7b14caf22ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:45.323358 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-3c9aef0c-12f5-48c7-b09c-7b14caf22ca6 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:45.326195 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-3c9aef0c-12f5-48c7-b09c-7b14caf22ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:45.332850 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1460/2917] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:45.337145 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7ecf3-f512-4bec-9180-939aee0e0335 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:45.347578 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7ecf3-f512-4bec-9180-939aee0e0335 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:45.352653 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f0d5c-ff55-4603-bb54-c1a3ad03ba43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:45.353327 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f0d5c-ff55-4603-bb54-c1a3ad03ba43 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:45.353611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f0d5c-ff55-4603-bb54-c1a3ad03ba43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:45.353884 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f0d5c-ff55-4603-bb54-c1a3ad03ba43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:45.354611 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f0d5c-ff55-4603-bb54-c1a3ad03ba43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:51:45.354982 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f0d5c-ff55-4603-bb54-c1a3ad03ba43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:45.356576 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f0d5c-ff55-4603-bb54-c1a3ad03ba43 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:45.368946 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1458/2918] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 16 15:51:45.373229 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9aef0c-12f5-48c7-b09c-7b14caf22ca6 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:45.382787 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9aef0c-12f5-48c7-b09c-7b14caf22ca6 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:45.387449 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f5d93-7266-472e-b0b1-9ad45d041c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:45.388094 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f5d93-7266-472e-b0b1-9ad45d041c84 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:45.388402 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f5d93-7266-472e-b0b1-9ad45d041c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:45.388675 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f5d93-7266-472e-b0b1-9ad45d041c84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:45.389360 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f5d93-7266-472e-b0b1-9ad45d041c84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:45.391331 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f5d93-7266-472e-b0b1-9ad45d041c84 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:45.531094 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1461/2919] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:51:45 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:51:45.535898 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f0d5c-ff55-4603-bb54-c1a3ad03ba43 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:45.546026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f0d5c-ff55-4603-bb54-c1a3ad03ba43 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:45.602041 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dfb57-a7cb-450a-b6cf-99530ec59495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:45.602700 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dfb57-a7cb-450a-b6cf-99530ec59495 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:45.603014 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dfb57-a7cb-450a-b6cf-99530ec59495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:45.603293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dfb57-a7cb-450a-b6cf-99530ec59495 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:45.603979 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61dfb57-a7cb-450a-b6cf-99530ec59495 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:45.605875 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61dfb57-a7cb-450a-b6cf-99530ec59495 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:45.715043 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1459/2920] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:51:45 2022] POST /identity/v3/users => generated 287 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:51:45.719017 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f5d93-7266-472e-b0b1-9ad45d041c84 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:45.730156 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f5d93-7266-472e-b0b1-9ad45d041c84 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:45.735073 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d38d7-63be-4391-b84b-d88324aefbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:45.735443 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d38d7-63be-4391-b84b-d88324aefbbc admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:45.735732 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d38d7-63be-4391-b84b-d88324aefbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:45.736009 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d38d7-63be-4391-b84b-d88324aefbbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:45.736778 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d38d7-63be-4391-b84b-d88324aefbbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:45.738352 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d38d7-63be-4391-b84b-d88324aefbbc admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:45.748668 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1462/2921] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:45.753031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dfb57-a7cb-450a-b6cf-99530ec59495 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:45.763109 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dfb57-a7cb-450a-b6cf-99530ec59495 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:45.767710 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc68b1-43ef-4993-848e-3aa188c1d1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:45.769077 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc68b1-43ef-4993-848e-3aa188c1d1ad admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:45.769375 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc68b1-43ef-4993-848e-3aa188c1d1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:45.769650 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc68b1-43ef-4993-848e-3aa188c1d1ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:45.770365 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc68b1-43ef-4993-848e-3aa188c1d1ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:45.771979 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc68b1-43ef-4993-848e-3aa188c1d1ad admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:45.782512 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1460/2922] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:45.786595 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d38d7-63be-4391-b84b-d88324aefbbc admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:45.796714 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d38d7-63be-4391-b84b-d88324aefbbc admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:45.801475 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73526dca-8307-4d82-aade-8ca70ac4842f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:45.802338 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73526dca-8307-4d82-aade-8ca70ac4842f admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:45.802618 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73526dca-8307-4d82-aade-8ca70ac4842f admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:45.802915 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73526dca-8307-4d82-aade-8ca70ac4842f admin admin] PATH_INFO: `/v3/projects/599d033c78814612b3eef131d6a86cac/users/de85d68db20b445c926b60be435dd907/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:45.805089 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:45.818323 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73526dca-8307-4d82-aade-8ca70ac4842f admin admin] RBAC: Authorizing `identity:create_grant(project_id=599d033c78814612b3eef131d6a86cac, user_id=de85d68db20b445c926b60be435dd907, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:45.821046 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73526dca-8307-4d82-aade-8ca70ac4842f admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:45.912296 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1463/2923] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:51:45 2022] PUT /identity/v3/projects/599d033c78814612b3eef131d6a86cac/users/de85d68db20b445c926b60be435dd907/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:45.922777 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db430eb6-43cc-422f-b994-e56f0c1aa09a None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:45.923094 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db430eb6-43cc-422f-b994-e56f0c1aa09a None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:45.923397 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db430eb6-43cc-422f-b994-e56f0c1aa09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:45.950701 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-db430eb6-43cc-422f-b994-e56f0c1aa09a None None] MFA Rules not processed for user `de85d68db20b445c926b60be435dd907`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:45.951944 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-db430eb6-43cc-422f-b994-e56f0c1aa09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:46.018871 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1461/2924] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:46.029790 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73526dca-8307-4d82-aade-8ca70ac4842f admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:46.031579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-73526dca-8307-4d82-aade-8ca70ac4842f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:46.089684 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73526dca-8307-4d82-aade-8ca70ac4842f admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:46.093349 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10782e53-831a-4aa7-9a36-f001bfc6dc9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:46.093994 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10782e53-831a-4aa7-9a36-f001bfc6dc9b service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:46.094276 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10782e53-831a-4aa7-9a36-f001bfc6dc9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:46.094551 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10782e53-831a-4aa7-9a36-f001bfc6dc9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:46.103512 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10782e53-831a-4aa7-9a36-f001bfc6dc9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:46.104948 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10782e53-831a-4aa7-9a36-f001bfc6dc9b service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:46.114315 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1464/2925] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:51:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:51:47.797569 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f61cf3-d4ef-4eaf-84b1-c967eef81f25 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:47.800891 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f61cf3-d4ef-4eaf-84b1-c967eef81f25 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:47.800891 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f61cf3-d4ef-4eaf-84b1-c967eef81f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:47.817308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c0f61cf3-d4ef-4eaf-84b1-c967eef81f25 None None] MFA Rules not processed for user `a4f499707998403d8f833eb783599466`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:47.818483 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c0f61cf3-d4ef-4eaf-84b1-c967eef81f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:47.831566 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:47.882625 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1462/2926] 10.222.0.27 () {58 vars in 967 bytes} [Sat Jul 16 15:51:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:47.970213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10782e53-831a-4aa7-9a36-f001bfc6dc9b service cinder] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:47.970213 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-10782e53-831a-4aa7-9a36-f001bfc6dc9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:47.998562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10782e53-831a-4aa7-9a36-f001bfc6dc9b service cinder] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:48.004263 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34169e22-be5f-4832-9ea4-f9bdb39dbef2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1464c83fb294a3d90e5763ef648e834', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:48.004930 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34169e22-be5f-4832-9ea4-f9bdb39dbef2 service swift] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:48.005229 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34169e22-be5f-4832-9ea4-f9bdb39dbef2 service swift] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:48.005487 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34169e22-be5f-4832-9ea4-f9bdb39dbef2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:48.013731 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34169e22-be5f-4832-9ea4-f9bdb39dbef2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:48.015382 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34169e22-be5f-4832-9ea4-f9bdb39dbef2 service swift] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:48.024902 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1465/2927] 10.222.0.27 () {60 vars in 1384 bytes} [Sat Jul 16 15:51:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 16 15:51:50.414765 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f61cf3-d4ef-4eaf-84b1-c967eef81f25 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:50.479498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f61cf3-d4ef-4eaf-84b1-c967eef81f25 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:50.483922 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380dd6d-b38d-4629-a2e4-46d9b6df8a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:50.484796 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380dd6d-b38d-4629-a2e4-46d9b6df8a11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:50.485073 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380dd6d-b38d-4629-a2e4-46d9b6df8a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:50.485371 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380dd6d-b38d-4629-a2e4-46d9b6df8a11 admin admin] PATH_INFO: `/v3/users/de85d68db20b445c926b60be435dd907` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:50.486569 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380dd6d-b38d-4629-a2e4-46d9b6df8a11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de85d68db20b445c926b60be435dd907)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:50.488576 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380dd6d-b38d-4629-a2e4-46d9b6df8a11 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:50.644422 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3380dd6d-b38d-4629-a2e4-46d9b6df8a11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de85d68db20b445c926b60be435dd907'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:50.731359 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3380dd6d-b38d-4629-a2e4-46d9b6df8a11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de85d68db20b445c926b60be435dd907'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:50.732090 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3380dd6d-b38d-4629-a2e4-46d9b6df8a11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de85d68db20b445c926b60be435dd907'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:50.733133 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3380dd6d-b38d-4629-a2e4-46d9b6df8a11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de85d68db20b445c926b60be435dd907'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:50.741707 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-3380dd6d-b38d-4629-a2e4-46d9b6df8a11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de85d68db20b445c926b60be435dd907'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:50.762690 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1463/2928] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:51:50 2022] DELETE /identity/v3/users/de85d68db20b445c926b60be435dd907 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:50.767124 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa81bce-23e2-4274-b8c2-6641a82cd6cb None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:50.767652 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa81bce-23e2-4274-b8c2-6641a82cd6cb None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:50.767982 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa81bce-23e2-4274-b8c2-6641a82cd6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:50.788084 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-eaa81bce-23e2-4274-b8c2-6641a82cd6cb None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:50.790337 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-eaa81bce-23e2-4274-b8c2-6641a82cd6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:50.859933 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1466/2929] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:50.871018 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380dd6d-b38d-4629-a2e4-46d9b6df8a11 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:50.872606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-3380dd6d-b38d-4629-a2e4-46d9b6df8a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:50.926388 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:50.939128 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380dd6d-b38d-4629-a2e4-46d9b6df8a11 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:50.943244 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fb9e0-da2f-401c-8c88-14b0e69df5d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:50.943900 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fb9e0-da2f-401c-8c88-14b0e69df5d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:50.944179 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fb9e0-da2f-401c-8c88-14b0e69df5d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:50.944451 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fb9e0-da2f-401c-8c88-14b0e69df5d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:50.953980 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fb9e0-da2f-401c-8c88-14b0e69df5d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:50.955467 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fb9e0-da2f-401c-8c88-14b0e69df5d2 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:50.966747 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1464/2930] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:51:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:51:51.442515 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa81bce-23e2-4274-b8c2-6641a82cd6cb None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:51.444651 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-eaa81bce-23e2-4274-b8c2-6641a82cd6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:51.458488 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa81bce-23e2-4274-b8c2-6641a82cd6cb None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:51.463531 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fbb2c-c271-4fd0-9874-ce0ee6820fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:51.464438 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fbb2c-c271-4fd0-9874-ce0ee6820fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:51.464835 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fbb2c-c271-4fd0-9874-ce0ee6820fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:51.465106 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fbb2c-c271-4fd0-9874-ce0ee6820fde admin admin] PATH_INFO: `/v3/projects/599d033c78814612b3eef131d6a86cac` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:51.466972 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17fbb2c-c271-4fd0-9874-ce0ee6820fde admin admin] RBAC: Authorizing `identity:delete_project(project_id=599d033c78814612b3eef131d6a86cac)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:51.473903 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17fbb2c-c271-4fd0-9874-ce0ee6820fde admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:51.480355 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:51.568570 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c17fbb2c-c271-4fd0-9874-ce0ee6820fde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '599d033c78814612b3eef131d6a86cac', 'request_id': 'req-c17fbb2c-c271-4fd0-9874-ce0ee6820fde'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:51.574541 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c17fbb2c-c271-4fd0-9874-ce0ee6820fde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '599d033c78814612b3eef131d6a86cac', 'request_id': 'req-c17fbb2c-c271-4fd0-9874-ce0ee6820fde'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:51.581593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c17fbb2c-c271-4fd0-9874-ce0ee6820fde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '599d033c78814612b3eef131d6a86cac', 'request_id': 'req-c17fbb2c-c271-4fd0-9874-ce0ee6820fde'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:51.655208 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c17fbb2c-c271-4fd0-9874-ce0ee6820fde admin admin] The token cache is being invalidate because project 599d033c78814612b3eef131d6a86cac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:51:51.658054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c17fbb2c-c271-4fd0-9874-ce0ee6820fde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:51.658054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-c17fbb2c-c271-4fd0-9874-ce0ee6820fde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:51.658054 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1467/2931] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:51:51 2022] DELETE /identity/v3/projects/599d033c78814612b3eef131d6a86cac => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:51.663195 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f18abb-e451-4749-937b-090f3208d9b4 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:51.663672 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f18abb-e451-4749-937b-090f3208d9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:51.664002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f18abb-e451-4749-937b-090f3208d9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:51.683114 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-e9f18abb-e451-4749-937b-090f3208d9b4 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:51.686794 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e9f18abb-e451-4749-937b-090f3208d9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:51.757096 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1465/2932] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:51.766796 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49a6f5-efb5-4d7f-ad1d-deed69bb8595 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:51.772682 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49a6f5-efb5-4d7f-ad1d-deed69bb8595 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:51.772682 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49a6f5-efb5-4d7f-ad1d-deed69bb8595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:51.787867 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-ee49a6f5-efb5-4d7f-ad1d-deed69bb8595 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:51.789067 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-ee49a6f5-efb5-4d7f-ad1d-deed69bb8595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:51.797295 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1468/2933] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:51.804351 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f18abb-e451-4749-937b-090f3208d9b4 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:51.814804 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f18abb-e451-4749-937b-090f3208d9b4 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:51.820261 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45baba2e-273a-48eb-82da-3bbafcb27982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:51.820925 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45baba2e-273a-48eb-82da-3bbafcb27982 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:51.821210 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45baba2e-273a-48eb-82da-3bbafcb27982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:51.821489 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45baba2e-273a-48eb-82da-3bbafcb27982 admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:51.822220 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45baba2e-273a-48eb-82da-3bbafcb27982 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:51:51.822525 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45baba2e-273a-48eb-82da-3bbafcb27982 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:51.824154 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45baba2e-273a-48eb-82da-3bbafcb27982 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:51.838350 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1466/2934] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:51:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:51:51.842816 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49a6f5-efb5-4d7f-ad1d-deed69bb8595 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:51.852336 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49a6f5-efb5-4d7f-ad1d-deed69bb8595 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:51.857492 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49884293-291d-446e-9714-bf4a3072f606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:51.858142 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49884293-291d-446e-9714-bf4a3072f606 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:51.858429 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49884293-291d-446e-9714-bf4a3072f606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:51.858855 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49884293-291d-446e-9714-bf4a3072f606 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:51.859583 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49884293-291d-446e-9714-bf4a3072f606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:51.861516 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49884293-291d-446e-9714-bf4a3072f606 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:51.939701 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1469/2935] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:51:51 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:51:51.944384 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45baba2e-273a-48eb-82da-3bbafcb27982 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:51.954445 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45baba2e-273a-48eb-82da-3bbafcb27982 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:52.011997 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94bb90-7926-4cd2-8f15-4ff28b687612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:52.012594 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94bb90-7926-4cd2-8f15-4ff28b687612 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:52.012876 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94bb90-7926-4cd2-8f15-4ff28b687612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:52.013152 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c94bb90-7926-4cd2-8f15-4ff28b687612 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:52.013844 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c94bb90-7926-4cd2-8f15-4ff28b687612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:52.015791 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c94bb90-7926-4cd2-8f15-4ff28b687612 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:52.126412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1467/2936] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:51:51 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 16 15:51:52.128327 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49884293-291d-446e-9714-bf4a3072f606 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:52.139286 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49884293-291d-446e-9714-bf4a3072f606 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:52.144833 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788dc07c-a427-42ad-a7fd-2a92b717cae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:52.145501 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788dc07c-a427-42ad-a7fd-2a92b717cae8 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:52.145781 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788dc07c-a427-42ad-a7fd-2a92b717cae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:52.146057 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788dc07c-a427-42ad-a7fd-2a92b717cae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:52.146782 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788dc07c-a427-42ad-a7fd-2a92b717cae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:52.148579 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788dc07c-a427-42ad-a7fd-2a92b717cae8 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:52.160801 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1470/2937] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:52.164861 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94bb90-7926-4cd2-8f15-4ff28b687612 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:52.185402 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c94bb90-7926-4cd2-8f15-4ff28b687612 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:52.191117 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b9b82-a85e-4090-8d6e-d4c69b92fa27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:52.191830 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b9b82-a85e-4090-8d6e-d4c69b92fa27 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:52.192126 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b9b82-a85e-4090-8d6e-d4c69b92fa27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:52.192407 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b9b82-a85e-4090-8d6e-d4c69b92fa27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:52.193149 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b9b82-a85e-4090-8d6e-d4c69b92fa27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:52.194796 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b9b82-a85e-4090-8d6e-d4c69b92fa27 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:52.219107 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1468/2938] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:51:52.219970 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788dc07c-a427-42ad-a7fd-2a92b717cae8 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:52.231411 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788dc07c-a427-42ad-a7fd-2a92b717cae8 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:52.236923 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0c16f-0936-41f8-b6e6-ea78e507c3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:52.246059 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0c16f-0936-41f8-b6e6-ea78e507c3f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:52.246059 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0c16f-0936-41f8-b6e6-ea78e507c3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:52.246059 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0c16f-0936-41f8-b6e6-ea78e507c3f8 admin admin] PATH_INFO: `/v3/projects/594c03dcc46741a481eaadf7a0982533/users/f458278f39cb452c8d7c7d5c30b7e473/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:52.258440 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0c16f-0936-41f8-b6e6-ea78e507c3f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=594c03dcc46741a481eaadf7a0982533, user_id=f458278f39cb452c8d7c7d5c30b7e473, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:52.261270 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0c16f-0936-41f8-b6e6-ea78e507c3f8 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:52.343165 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1471/2939] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:51:52 2022] PUT /identity/v3/projects/594c03dcc46741a481eaadf7a0982533/users/f458278f39cb452c8d7c7d5c30b7e473/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 16 15:51:52.350865 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71984be-9c58-4220-a3d7-83e8ef5968be None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:52.351192 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71984be-9c58-4220-a3d7-83e8ef5968be None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:52.351475 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71984be-9c58-4220-a3d7-83e8ef5968be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:52.384245 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c71984be-9c58-4220-a3d7-83e8ef5968be None None] MFA Rules not processed for user `f458278f39cb452c8d7c7d5c30b7e473`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:51:52.385497 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c71984be-9c58-4220-a3d7-83e8ef5968be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:52.445556 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:52.479398 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1469/2940] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:51:52 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:51:52.492577 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0c16f-0936-41f8-b6e6-ea78e507c3f8 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:52.493609 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-bee0c16f-0936-41f8-b6e6-ea78e507c3f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:52.563376 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0c16f-0936-41f8-b6e6-ea78e507c3f8 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:52.567808 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db851cc0-4dcb-48ff-92c0-af6304e6e17f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d689162291f4f4b97e886ea20d3a8b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:52.568452 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db851cc0-4dcb-48ff-92c0-af6304e6e17f service glance] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:52.573345 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db851cc0-4dcb-48ff-92c0-af6304e6e17f service glance] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:52.573345 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db851cc0-4dcb-48ff-92c0-af6304e6e17f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:52.578676 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db851cc0-4dcb-48ff-92c0-af6304e6e17f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:52.580160 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db851cc0-4dcb-48ff-92c0-af6304e6e17f service glance] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:52.590553 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1472/2941] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:51:52 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:51:56.865299 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71984be-9c58-4220-a3d7-83e8ef5968be None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:56.866771 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c71984be-9c58-4220-a3d7-83e8ef5968be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:56.936892 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71984be-9c58-4220-a3d7-83e8ef5968be None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:56.942023 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba0d89-225f-4d92-b8cb-5c5aef846d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:56.942916 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba0d89-225f-4d92-b8cb-5c5aef846d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:56.943205 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba0d89-225f-4d92-b8cb-5c5aef846d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:56.943485 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba0d89-225f-4d92-b8cb-5c5aef846d65 admin admin] PATH_INFO: `/v3/users/745d930ebbab403c9e974f47d3719035` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:56.944716 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba0d89-225f-4d92-b8cb-5c5aef846d65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=745d930ebbab403c9e974f47d3719035)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:56.946795 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba0d89-225f-4d92-b8cb-5c5aef846d65 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:57.132666 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-59ba0d89-225f-4d92-b8cb-5c5aef846d65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '745d930ebbab403c9e974f47d3719035'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:57.191412 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-59ba0d89-225f-4d92-b8cb-5c5aef846d65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '745d930ebbab403c9e974f47d3719035'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:57.192167 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-59ba0d89-225f-4d92-b8cb-5c5aef846d65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '745d930ebbab403c9e974f47d3719035'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:57.192779 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-59ba0d89-225f-4d92-b8cb-5c5aef846d65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '745d930ebbab403c9e974f47d3719035'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:57.208499 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-59ba0d89-225f-4d92-b8cb-5c5aef846d65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '745d930ebbab403c9e974f47d3719035'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:57.220658 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1470/2942] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:51:56 2022] DELETE /identity/v3/users/745d930ebbab403c9e974f47d3719035 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:57.480308 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db851cc0-4dcb-48ff-92c0-af6304e6e17f service glance] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:57.495237 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db851cc0-4dcb-48ff-92c0-af6304e6e17f service glance] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:57.531391 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:57.554300 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d7ca8-1213-4468-bfd3-7621a9b1fb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:57.555188 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d7ca8-1213-4468-bfd3-7621a9b1fb30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:57.555471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d7ca8-1213-4468-bfd3-7621a9b1fb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:57.555741 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d7ca8-1213-4468-bfd3-7621a9b1fb30 admin admin] PATH_INFO: `/v3/projects/d5509d64a1aa472095ad35da222f9963` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:57.556913 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d7ca8-1213-4468-bfd3-7621a9b1fb30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5509d64a1aa472095ad35da222f9963)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:57.558808 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d7ca8-1213-4468-bfd3-7621a9b1fb30 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:57.662648 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-b38d7ca8-1213-4468-bfd3-7621a9b1fb30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5509d64a1aa472095ad35da222f9963', 'request_id': 'req-b38d7ca8-1213-4468-bfd3-7621a9b1fb30'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:57.668428 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-b38d7ca8-1213-4468-bfd3-7621a9b1fb30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5509d64a1aa472095ad35da222f9963', 'request_id': 'req-b38d7ca8-1213-4468-bfd3-7621a9b1fb30'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:57.675085 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-b38d7ca8-1213-4468-bfd3-7621a9b1fb30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5509d64a1aa472095ad35da222f9963', 'request_id': 'req-b38d7ca8-1213-4468-bfd3-7621a9b1fb30'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:57.739452 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-b38d7ca8-1213-4468-bfd3-7621a9b1fb30 admin admin] The token cache is being invalidate because project d5509d64a1aa472095ad35da222f9963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:51:57.739740 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-b38d7ca8-1213-4468-bfd3-7621a9b1fb30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:57.740491 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-b38d7ca8-1213-4468-bfd3-7621a9b1fb30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:57.742122 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1473/2943] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:51:57 2022] DELETE /identity/v3/projects/d5509d64a1aa472095ad35da222f9963 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:57.746614 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba0d89-225f-4d92-b8cb-5c5aef846d65 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:57.756554 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-59ba0d89-225f-4d92-b8cb-5c5aef846d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:57.762775 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba0d89-225f-4d92-b8cb-5c5aef846d65 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:57.823416 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65e229-fcd2-4b9f-8789-6e155172f8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:57.824357 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65e229-fcd2-4b9f-8789-6e155172f8bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:57.824635 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65e229-fcd2-4b9f-8789-6e155172f8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:57.824908 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65e229-fcd2-4b9f-8789-6e155172f8bf admin admin] PATH_INFO: `/v3/users/365f84b02d354f5bbfc42a46b33b4825` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:57.826095 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65e229-fcd2-4b9f-8789-6e155172f8bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=365f84b02d354f5bbfc42a46b33b4825)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:57.828036 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65e229-fcd2-4b9f-8789-6e155172f8bf admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:57.832705 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:57.990479 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6d65e229-fcd2-4b9f-8789-6e155172f8bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '365f84b02d354f5bbfc42a46b33b4825'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:58.062503 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6d65e229-fcd2-4b9f-8789-6e155172f8bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '365f84b02d354f5bbfc42a46b33b4825'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:58.063148 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6d65e229-fcd2-4b9f-8789-6e155172f8bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '365f84b02d354f5bbfc42a46b33b4825'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:58.063754 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6d65e229-fcd2-4b9f-8789-6e155172f8bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '365f84b02d354f5bbfc42a46b33b4825'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:58.071557 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-6d65e229-fcd2-4b9f-8789-6e155172f8bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '365f84b02d354f5bbfc42a46b33b4825'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:58.090791 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1471/2944] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:51:57 2022] DELETE /identity/v3/users/365f84b02d354f5bbfc42a46b33b4825 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:51:58.387540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d7ca8-1213-4468-bfd3-7621a9b1fb30 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:51:58.388948 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b38d7ca8-1213-4468-bfd3-7621a9b1fb30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:51:58.404661 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d7ca8-1213-4468-bfd3-7621a9b1fb30 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:51:58.464804 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169bf4b-44b4-42af-8824-c220e15a9a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:51:58.466183 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169bf4b-44b4-42af-8824-c220e15a9a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:51:58.466471 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169bf4b-44b4-42af-8824-c220e15a9a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:51:58.466962 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169bf4b-44b4-42af-8824-c220e15a9a9e admin admin] PATH_INFO: `/v3/projects/fe4f9e430c484aa9aa4fd8e27db0918e` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:51:58.468281 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169bf4b-44b4-42af-8824-c220e15a9a9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe4f9e430c484aa9aa4fd8e27db0918e)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:51:58.470222 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169bf4b-44b4-42af-8824-c220e15a9a9e admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:51:58.578238 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a169bf4b-44b4-42af-8824-c220e15a9a9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe4f9e430c484aa9aa4fd8e27db0918e', 'request_id': 'req-a169bf4b-44b4-42af-8824-c220e15a9a9e'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:58.584637 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a169bf4b-44b4-42af-8824-c220e15a9a9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe4f9e430c484aa9aa4fd8e27db0918e', 'request_id': 'req-a169bf4b-44b4-42af-8824-c220e15a9a9e'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:58.590851 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a169bf4b-44b4-42af-8824-c220e15a9a9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe4f9e430c484aa9aa4fd8e27db0918e', 'request_id': 'req-a169bf4b-44b4-42af-8824-c220e15a9a9e'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:58.666495 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:51:58.671206 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a169bf4b-44b4-42af-8824-c220e15a9a9e admin admin] The token cache is being invalidate because project fe4f9e430c484aa9aa4fd8e27db0918e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:51:58.671493 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a169bf4b-44b4-42af-8824-c220e15a9a9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:58.672128 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-a169bf4b-44b4-42af-8824-c220e15a9a9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:51:58.673671 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1474/2945] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:51:58 2022] DELETE /identity/v3/projects/fe4f9e430c484aa9aa4fd8e27db0918e => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:52:08.086695 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=26,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:52:08.807956 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:52:28.753992 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65e229-fcd2-4b9f-8789-6e155172f8bf admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:52:28.755599 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-6d65e229-fcd2-4b9f-8789-6e155172f8bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:52:28.769364 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65e229-fcd2-4b9f-8789-6e155172f8bf admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:52:28.825211 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34573d-67c6-45c2-b68d-260ee47cf14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:52:28.826069 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34573d-67c6-45c2-b68d-260ee47cf14f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:52:28.826345 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34573d-67c6-45c2-b68d-260ee47cf14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:52:28.826617 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34573d-67c6-45c2-b68d-260ee47cf14f admin admin] PATH_INFO: `/v3/users/f458278f39cb452c8d7c7d5c30b7e473` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:52:28.827818 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34573d-67c6-45c2-b68d-260ee47cf14f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f458278f39cb452c8d7c7d5c30b7e473)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:52:28.829706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34573d-67c6-45c2-b68d-260ee47cf14f admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:52:29.029673 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-dc34573d-67c6-45c2-b68d-260ee47cf14f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f458278f39cb452c8d7c7d5c30b7e473'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:52:29.116014 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-dc34573d-67c6-45c2-b68d-260ee47cf14f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f458278f39cb452c8d7c7d5c30b7e473'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:52:29.117437 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-dc34573d-67c6-45c2-b68d-260ee47cf14f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f458278f39cb452c8d7c7d5c30b7e473'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:52:29.117900 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-dc34573d-67c6-45c2-b68d-260ee47cf14f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f458278f39cb452c8d7c7d5c30b7e473'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:52:29.125189 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-dc34573d-67c6-45c2-b68d-260ee47cf14f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f458278f39cb452c8d7c7d5c30b7e473'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:52:29.142468 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1472/2946] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:52:28 2022] DELETE /identity/v3/users/f458278f39cb452c8d7c7d5c30b7e473 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:52:29.147092 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2318a-ab86-4970-b78c-47940d672811 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:52:29.147577 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2318a-ab86-4970-b78c-47940d672811 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:52:29.147680 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2318a-ab86-4970-b78c-47940d672811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:52:29.166081 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-16e2318a-ab86-4970-b78c-47940d672811 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:52:29.167320 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-16e2318a-ab86-4970-b78c-47940d672811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:52:29.225105 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1475/2947] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:52:29.235308 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34573d-67c6-45c2-b68d-260ee47cf14f admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:52:29.236739 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-dc34573d-67c6-45c2-b68d-260ee47cf14f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:52:29.303623 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34573d-67c6-45c2-b68d-260ee47cf14f admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:52:29.308053 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a1229-5e83-4903-b101-4890f8319d18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:52:29.308694 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a1229-5e83-4903-b101-4890f8319d18 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:52:29.309017 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a1229-5e83-4903-b101-4890f8319d18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:52:29.309293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a1229-5e83-4903-b101-4890f8319d18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:52:29.318629 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a1229-5e83-4903-b101-4890f8319d18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:52:29.320151 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a1229-5e83-4903-b101-4890f8319d18 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:52:29.329690 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1473/2948] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:52:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:52:29.741172 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2318a-ab86-4970-b78c-47940d672811 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:52:29.743287 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-16e2318a-ab86-4970-b78c-47940d672811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:52:29.755683 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2318a-ab86-4970-b78c-47940d672811 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:52:29.761983 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc15c1c-c9bb-44ef-9bd4-443b43b62061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:52:29.762962 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc15c1c-c9bb-44ef-9bd4-443b43b62061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:52:29.763243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc15c1c-c9bb-44ef-9bd4-443b43b62061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:52:29.763519 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc15c1c-c9bb-44ef-9bd4-443b43b62061 admin admin] PATH_INFO: `/v3/projects/594c03dcc46741a481eaadf7a0982533` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:52:29.764728 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc15c1c-c9bb-44ef-9bd4-443b43b62061 admin admin] RBAC: Authorizing `identity:delete_project(project_id=594c03dcc46741a481eaadf7a0982533)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:52:29.766640 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc15c1c-c9bb-44ef-9bd4-443b43b62061 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:52:29.911898 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-fbc15c1c-c9bb-44ef-9bd4-443b43b62061 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '594c03dcc46741a481eaadf7a0982533', 'request_id': 'req-fbc15c1c-c9bb-44ef-9bd4-443b43b62061'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:52:29.917445 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-fbc15c1c-c9bb-44ef-9bd4-443b43b62061 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '594c03dcc46741a481eaadf7a0982533', 'request_id': 'req-fbc15c1c-c9bb-44ef-9bd4-443b43b62061'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:52:29.923549 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-fbc15c1c-c9bb-44ef-9bd4-443b43b62061 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '594c03dcc46741a481eaadf7a0982533', 'request_id': 'req-fbc15c1c-c9bb-44ef-9bd4-443b43b62061'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:52:30.001612 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-fbc15c1c-c9bb-44ef-9bd4-443b43b62061 admin admin] The token cache is being invalidate because project 594c03dcc46741a481eaadf7a0982533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:52:30.001895 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-fbc15c1c-c9bb-44ef-9bd4-443b43b62061 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:52:30.002536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-fbc15c1c-c9bb-44ef-9bd4-443b43b62061 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:52:30.004118 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1476/2949] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:52:29 2022] DELETE /identity/v3/projects/594c03dcc46741a481eaadf7a0982533 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:52:30.009801 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b722a-7ea3-4575-af20-30dc3406b396 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:52:30.010341 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b722a-7ea3-4575-af20-30dc3406b396 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:52:30.010440 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b722a-7ea3-4575-af20-30dc3406b396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:52:30.029781 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-c35b722a-7ea3-4575-af20-30dc3406b396 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:52:30.034498 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-c35b722a-7ea3-4575-af20-30dc3406b396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:52:30.040095 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:52:30.091158 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1474/2950] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:52:30.099009 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e955c2-99da-4fc7-b8d4-a0935e459fc4 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:52:30.099312 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e955c2-99da-4fc7-b8d4-a0935e459fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:52:30.099582 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e955c2-99da-4fc7-b8d4-a0935e459fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:52:30.117659 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-b8e955c2-99da-4fc7-b8d4-a0935e459fc4 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:52:30.118978 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-b8e955c2-99da-4fc7-b8d4-a0935e459fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:52:30.126831 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1477/2951] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:52:30.131031 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b722a-7ea3-4575-af20-30dc3406b396 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:52:30.142615 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b722a-7ea3-4575-af20-30dc3406b396 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:52:30.147723 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b647458-09b6-4687-a726-9cc6ad0f48bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:52:30.148628 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b647458-09b6-4687-a726-9cc6ad0f48bb admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:52:30.148912 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b647458-09b6-4687-a726-9cc6ad0f48bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:52:30.149235 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b647458-09b6-4687-a726-9cc6ad0f48bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:52:30.150814 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b647458-09b6-4687-a726-9cc6ad0f48bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80256) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 16 15:52:30.158149 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b647458-09b6-4687-a726-9cc6ad0f48bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:52:30.158149 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b647458-09b6-4687-a726-9cc6ad0f48bb admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:52:30.164066 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1475/2952] 10.222.0.27 () {58 vars in 1152 bytes} [Sat Jul 16 15:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 16 15:52:30.168517 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e955c2-99da-4fc7-b8d4-a0935e459fc4 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:52:30.178031 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e955c2-99da-4fc7-b8d4-a0935e459fc4 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:52:30.183364 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f640df-bec4-40ac-9b2f-536693896b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:52:30.184006 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f640df-bec4-40ac-9b2f-536693896b32 admin admin] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:52:30.184287 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f640df-bec4-40ac-9b2f-536693896b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:52:30.184562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f640df-bec4-40ac-9b2f-536693896b32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:52:30.185366 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f640df-bec4-40ac-9b2f-536693896b32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:52:30.187304 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f640df-bec4-40ac-9b2f-536693896b32 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:52:30.271386 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1478/2953] 10.222.0.27 () {60 vars in 1152 bytes} [Sat Jul 16 15:52:30 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:52:30.275901 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b647458-09b6-4687-a726-9cc6ad0f48bb admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:52:30.286194 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b647458-09b6-4687-a726-9cc6ad0f48bb admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:52:30.344002 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3795ee-b61a-4754-9cf7-330e4d9f49c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:52:30.344670 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3795ee-b61a-4754-9cf7-330e4d9f49c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:52:30.344950 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3795ee-b61a-4754-9cf7-330e4d9f49c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:52:30.345236 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3795ee-b61a-4754-9cf7-330e4d9f49c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:52:30.345910 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3795ee-b61a-4754-9cf7-330e4d9f49c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:52:30.347846 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3795ee-b61a-4754-9cf7-330e4d9f49c6 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:52:30.458954 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1476/2954] 10.222.0.27 () {60 vars in 1143 bytes} [Sat Jul 16 15:52:30 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 16 15:52:30.462963 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f640df-bec4-40ac-9b2f-536693896b32 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:52:30.473930 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f640df-bec4-40ac-9b2f-536693896b32 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:52:30.479288 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d84da1-5c95-4885-9ad6-423a3f4c5029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:52:30.479957 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d84da1-5c95-4885-9ad6-423a3f4c5029 admin admin] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:52:30.480243 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d84da1-5c95-4885-9ad6-423a3f4c5029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:52:30.480615 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d84da1-5c95-4885-9ad6-423a3f4c5029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:52:30.481346 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d84da1-5c95-4885-9ad6-423a3f4c5029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:52:30.483029 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d84da1-5c95-4885-9ad6-423a3f4c5029 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:52:30.494465 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1479/2955] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 16 15:52:30.498761 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3795ee-b61a-4754-9cf7-330e4d9f49c6 admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:52:30.509483 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3795ee-b61a-4754-9cf7-330e4d9f49c6 admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:52:30.513988 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4728593-37b3-4374-b57f-c4aca7d40991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:52:30.514644 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4728593-37b3-4374-b57f-c4aca7d40991 admin admin] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:52:30.515015 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4728593-37b3-4374-b57f-c4aca7d40991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:52:30.515293 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4728593-37b3-4374-b57f-c4aca7d40991 admin admin] PATH_INFO: `/v3/roles` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:52:30.516014 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4728593-37b3-4374-b57f-c4aca7d40991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:52:30.517594 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4728593-37b3-4374-b57f-c4aca7d40991 admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:52:30.528101 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1477/2956] 10.222.0.27 () {58 vars in 1121 bytes} [Sat Jul 16 15:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 16 15:52:30.532024 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d84da1-5c95-4885-9ad6-423a3f4c5029 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:52:30.543012 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d84da1-5c95-4885-9ad6-423a3f4c5029 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:52:30.547717 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d728803-2aa8-4cf8-a4d2-e048c49ccdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:52:30.549865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d728803-2aa8-4cf8-a4d2-e048c49ccdd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:52:30.549865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d728803-2aa8-4cf8-a4d2-e048c49ccdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:52:30.549865 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d728803-2aa8-4cf8-a4d2-e048c49ccdd3 admin admin] PATH_INFO: `/v3/projects/5396730e7a494cbe832e3a59c28760da/users/bc72d3286f184a128eae8682d52929a6/roles/b7f92100b08f4dc6b2b6866a2955d2ab` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:52:30.565324 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d728803-2aa8-4cf8-a4d2-e048c49ccdd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5396730e7a494cbe832e3a59c28760da, user_id=bc72d3286f184a128eae8682d52929a6, role_id=b7f92100b08f4dc6b2b6866a2955d2ab)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:52:30.568092 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d728803-2aa8-4cf8-a4d2-e048c49ccdd3 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:52:30.650562 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1480/2957] 10.222.0.27 () {60 vars in 1482 bytes} [Sat Jul 16 15:52:30 2022] PUT /identity/v3/projects/5396730e7a494cbe832e3a59c28760da/users/bc72d3286f184a128eae8682d52929a6/roles/b7f92100b08f4dc6b2b6866a2955d2ab => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:52:30.661706 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732c4d5-f522-4c01-9b17-0fe5d7b7d571 None None] REQUEST_METHOD: `POST` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:52:30.661976 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732c4d5-f522-4c01-9b17-0fe5d7b7d571 None None] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:52:30.662254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732c4d5-f522-4c01-9b17-0fe5d7b7d571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:52:30.690525 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.auth.core [None req-0732c4d5-f522-4c01-9b17-0fe5d7b7d571 None None] MFA Rules not processed for user `bc72d3286f184a128eae8682d52929a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80256) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:52:30.691766 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-0732c4d5-f522-4c01-9b17-0fe5d7b7d571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:52:30.755730 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1478/2958] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:52:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 16 15:52:30.766688 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d728803-2aa8-4cf8-a4d2-e048c49ccdd3 admin admin] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:52:30.768396 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-9d728803-2aa8-4cf8-a4d2-e048c49ccdd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:52:30.791143 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:52:30.826326 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d728803-2aa8-4cf8-a4d2-e048c49ccdd3 admin admin] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:52:30.830574 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f6a45-9154-407d-a8e8-c56df32f309a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8c21d12c977e4c36b4f0ed442b11ef49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:52:30.831234 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f6a45-9154-407d-a8e8-c56df32f309a service cinder] REQUEST_METHOD: `GET` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:52:30.831517 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f6a45-9154-407d-a8e8-c56df32f309a service cinder] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:52:30.831792 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f6a45-9154-407d-a8e8-c56df32f309a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:52:30.841317 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f6a45-9154-407d-a8e8-c56df32f309a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:52:30.842739 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f6a45-9154-407d-a8e8-c56df32f309a service cinder] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:52:30.851540 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1481/2959] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:52:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:52:33.952097 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732c4d5-f522-4c01-9b17-0fe5d7b7d571 None None] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:52:33.961606 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:52:34.015512 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732c4d5-f522-4c01-9b17-0fe5d7b7d571 None None] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:52:34.019959 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe747c-f615-4a0c-be47-65401d3cf5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:52:34.020819 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe747c-f615-4a0c-be47-65401d3cf5bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:52:34.021096 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe747c-f615-4a0c-be47-65401d3cf5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:52:34.021366 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe747c-f615-4a0c-be47-65401d3cf5bb admin admin] PATH_INFO: `/v3/users/bc72d3286f184a128eae8682d52929a6` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:52:34.022545 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe747c-f615-4a0c-be47-65401d3cf5bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc72d3286f184a128eae8682d52929a6)` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:52:34.024744 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe747c-f615-4a0c-be47-65401d3cf5bb admin admin] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:52:34.171961 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8fbe747c-f615-4a0c-be47-65401d3cf5bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc72d3286f184a128eae8682d52929a6'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:52:34.255518 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8fbe747c-f615-4a0c-be47-65401d3cf5bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc72d3286f184a128eae8682d52929a6'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:52:34.256130 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8fbe747c-f615-4a0c-be47-65401d3cf5bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc72d3286f184a128eae8682d52929a6'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:52:34.256735 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8fbe747c-f615-4a0c-be47-65401d3cf5bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc72d3286f184a128eae8682d52929a6'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:52:34.263863 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.notifications [None req-8fbe747c-f615-4a0c-be47-65401d3cf5bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc72d3286f184a128eae8682d52929a6'} {{(pid=80256) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:52:34.281544 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1479/2960] 10.222.0.27 () {58 vars in 1223 bytes} [Sat Jul 16 15:52:33 2022] DELETE /identity/v3/users/bc72d3286f184a128eae8682d52929a6 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:52:34.286258 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6906d05-7e2b-4375-9ea6-cef5f7461d12 None None] REQUEST_METHOD: `POST` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:52:34.287820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6906d05-7e2b-4375-9ea6-cef5f7461d12 None None] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:52:34.287820 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6906d05-7e2b-4375-9ea6-cef5f7461d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:52:34.304393 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.auth.core [None req-a6906d05-7e2b-4375-9ea6-cef5f7461d12 None None] MFA Rules not processed for user `2e3ec4696fd447c89f8d404bb4540c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80257) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 16 15:52:34.305593 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a6906d05-7e2b-4375-9ea6-cef5f7461d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:52:34.362832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1482/2961] 10.222.0.27 () {58 vars in 957 bytes} [Sat Jul 16 15:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 16 15:52:34.373133 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe747c-f615-4a0c-be47-65401d3cf5bb admin admin] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:52:34.374528 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-8fbe747c-f615-4a0c-be47-65401d3cf5bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:52:34.430751 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe747c-f615-4a0c-be47-65401d3cf5bb admin admin] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:52:34.434836 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9591594-d882-449d-8638-0bea74cd6521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e372cc62af24664b01a967435eb8902', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:52:34.435509 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9591594-d882-449d-8638-0bea74cd6521 service neutron] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:52:34.435792 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9591594-d882-449d-8638-0bea74cd6521 service neutron] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:52:34.436067 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9591594-d882-449d-8638-0bea74cd6521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:52:34.445026 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9591594-d882-449d-8638-0bea74cd6521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:52:34.446445 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9591594-d882-449d-8638-0bea74cd6521 service neutron] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:52:34.456125 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1480/2962] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:52:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:52:34.865803 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6906d05-7e2b-4375-9ea6-cef5f7461d12 None None] Authenticating user token {{(pid=80257) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:52:34.869555 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.fernet_utils [None req-a6906d05-7e2b-4375-9ea6-cef5f7461d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80257) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:52:34.879346 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6906d05-7e2b-4375-9ea6-cef5f7461d12 None None] Validating token access rules against request {{(pid=80257) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:52:34.887832 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02685a-c162-4f7a-9f71-f296bd2fcd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e3ec4696fd447c89f8d404bb4540c6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '094ab8f948f540f7a205e832cbae4778', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80257) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:52:34.888877 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02685a-c162-4f7a-9f71-f296bd2fcd64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:52:34.889156 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02685a-c162-4f7a-9f71-f296bd2fcd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:52:34.889431 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02685a-c162-4f7a-9f71-f296bd2fcd64 admin admin] PATH_INFO: `/v3/projects/5396730e7a494cbe832e3a59c28760da` {{(pid=80257) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:52:34.890806 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02685a-c162-4f7a-9f71-f296bd2fcd64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5396730e7a494cbe832e3a59c28760da)` {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:52:34.892939 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02685a-c162-4f7a-9f71-f296bd2fcd64 admin admin] RBAC: Authorization granted {{(pid=80257) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:52:35.001536 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-cb02685a-c162-4f7a-9f71-f296bd2fcd64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5396730e7a494cbe832e3a59c28760da', 'request_id': 'req-cb02685a-c162-4f7a-9f71-f296bd2fcd64'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:52:35.007004 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-cb02685a-c162-4f7a-9f71-f296bd2fcd64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5396730e7a494cbe832e3a59c28760da', 'request_id': 'req-cb02685a-c162-4f7a-9f71-f296bd2fcd64'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:52:35.013692 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-cb02685a-c162-4f7a-9f71-f296bd2fcd64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5396730e7a494cbe832e3a59c28760da', 'request_id': 'req-cb02685a-c162-4f7a-9f71-f296bd2fcd64'} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:52:35.093038 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-cb02685a-c162-4f7a-9f71-f296bd2fcd64 admin admin] The token cache is being invalidate because project 5396730e7a494cbe832e3a59c28760da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80257) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 16 15:52:35.093326 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-cb02685a-c162-4f7a-9f71-f296bd2fcd64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:52:35.093949 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG keystone.notifications [None req-cb02685a-c162-4f7a-9f71-f296bd2fcd64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80257) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 16 15:52:35.095553 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: [pid: 80257|app: 0|req: 1483/2963] 10.222.0.27 () {58 vars in 1232 bytes} [Sat Jul 16 15:52:34 2022] DELETE /identity/v3/projects/5396730e7a494cbe832e3a59c28760da => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 16 15:52:44.451915 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=75,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:52:45.089098 dv-cinder-836252-5-iscsi devstack@keystone.service[80257]: DEBUG dbcounter [-] [80257] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 15:54:16.876512 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9591594-d882-449d-8638-0bea74cd6521 service neutron] Authenticating user token {{(pid=80256) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 16 15:54:16.878013 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-e9591594-d882-449d-8638-0bea74cd6521 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:54:16.946970 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9591594-d882-449d-8638-0bea74cd6521 service neutron] Validating token access rules against request {{(pid=80256) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 16 15:54:17.006254 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa39d3-2791-43b7-b228-5a923bdfab59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a30cee766814c8685578819261705ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2739f00eb01e46a9acf90ffa360d7d8c', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80256) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 16 15:54:17.006919 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa39d3-2791-43b7-b228-5a923bdfab59 service placement] REQUEST_METHOD: `GET` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 16 15:54:17.007212 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa39d3-2791-43b7-b228-5a923bdfab59 service placement] SCRIPT_NAME: `/identity` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 16 15:54:17.007571 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa39d3-2791-43b7-b228-5a923bdfab59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80256) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 16 15:54:17.009424 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.fernet_utils [None req-bffa39d3-2791-43b7-b228-5a923bdfab59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80256) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 16 15:54:17.020890 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffa39d3-2791-43b7-b228-5a923bdfab59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 16 15:54:17.022460 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffa39d3-2791-43b7-b228-5a923bdfab59 service placement] RBAC: Authorization granted {{(pid=80256) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 16 15:54:17.032448 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: [pid: 80256|app: 0|req: 1481/2964] 10.222.0.27 () {60 vars in 1365 bytes} [Sat Jul 16 15:54:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 16 15:54:27.028044 dv-cinder-836252-5-iscsi devstack@keystone.service[80256]: DEBUG dbcounter [-] [80256] Writing DB stats keystone:SELECT=58 {{(pid=80256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}